[petsc-dev] Implementing longer pipelines with VecDotBegin and VecDotEnd

[Jed Brown]

Barry Smith <bsmith at mcs.anl.gov> writes:

>> On Mar 23, 2017, at 10:54 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
>> 
>> Barry Smith <bsmith at mcs.anl.gov> writes:
>>>> Meh,
>>>> 
>>>> VecNormBegin(X,&request1x);
>>>> VecNormBegin(Y,&request1y);
>>>> VecNormEnd(X,request1x,&norm);
>>>> VecAXPY(Y,-1,X);
>>>> VecNormBegin(Y,&request2y);
>>>> VecNormEnd(Y,request2y,&norm2y);
>>>> VecNormEnd(Y,request1y,&norm1y);
>>> 
>>>   I don't understand what you are getting at here. You don't seem to be understanding my use case where multiple inner products/norms share the same MPI communication (which was the original reason for VecNormBegin/End) see for example KSPSolve_CR
>>> 
>>>    Are you somehow (incompetently) saying that the first two VecNorms
>>>    somehow share the same parallel communication (even though they
>>>    have different request values) while the third Norm has its own
>>>    MPI communication. 
>> 
>> Yeah, same as now.  Every time you call *Begin() using a communicator,
>> you get a new request for something in that "batch".  When the batch is
>> closed, either by a *End() or PetscCommSplitReductionBegin(), any future
>> *Begin() calls will go into a new batch.
>
>    Ok, so to support their use case they would need to explicitly call PetscCommSplitReductionBegin?

Yes, just like all the other pipelined methods do now.

>    Start first inner product
>    Do mat mult
>     do PetscCommSplitReductionBegin() to indicate the first batch is done?
>     start second inner product (will be handled in a second batch)
>    do stuff
>    get result of first inner product
>    ....
>
>    Ok I can live with this model. But my "hoist" model would have, in some cases fewer objects managed by the user but it would require an explicit 
> PetscCommSplitReductionCreate().

Yeah, but that sucks if an outer scope wants to start a reduction
(perhaps a time integrator's CFL check) without changing the API of its
solver to plumb the reduction in.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20170323/44cac1c3/attachment.sig>