[mpich-discuss] Progress for user-defined non-blocking collectives

[Jed Brown]

Right, I don't have a problem with the implementation using a background
thread, but I don't want each library to have its own background thread.
(Because this could be quite a large number of threads for a coupled
multiphysics simulation.)

So if we need a common interface at this level, I'd at least like to
discuss doing it at the MPI level.
On Feb 21, 2012 6:04 PM, "Dave Goodell" <goodell at mcs.anl.gov> wrote:

> On Feb 21, 2012, at 4:56 PM CST, Jed Brown wrote:
>
> > Dave, it looks like you implemented the non-blocking collectives for
> MPI-3. How reasonable do you think it would be to expose enough hooks to be
> able to write a user-defined non-blocking collective that could make useful
> progress?
>
> I'm leery of doing this.  There are a reasonable number of assumptions in
> the NBC impl code that it really is only being used for internal
> nonblocking collectives which would make this sort of thing a bit of a
> pain.  But I don't think the technical issues are exceptionally difficult,
> just not really pleasant either.
>
> > This comes up pretty frequently where there is a high-level operation
> with collective semantics that internally needs to perform multiple
> dependent communications. We might still expose a non-blocking interface to
> the user, but the performance benefit is limited because the multiple
> rounds either take place up-front or at the end.
>
> OK, I believe I understand your use case and why you want this.  It seems
> that part of the problem is that you have chosen to expose a nonblocking
> interface but you don't want either (A) a background thread or (B) to
> require the callers of your interface to make intermediate progress calls
> to your library.
>
> > Generalized requests aren't (currently) a good solution because (from
> what I can tell), they only make progress when _that request_ is polled. In
> practice, you want to poke those requests from other library calls (or,
> eventually/on some systems, by a progress thread), just like MPI native
> operations can make progress without explicitly being polled.
>
>
> I just skimmed the generalized req code and the lack of implicit progress
> does seem to be the way it's been implemented.  I'm assuming you're
> thinking about the proposed generalized request extensions rather than the
> MPI-2 flavor, since they don't have a way for MPI to drive progress.
>
> Generalized requests seem more appropriate to this scenario if we can ask
> MPI to implicitly make progress on it.  Something like a
> "MPIX_Register_grequest" routine.  Also we need to ensure that you are
> allowed to make MPI calls from within the poll_fn or wait_fn.  I'm not
> certain this is actually safe right now.
>
> -Dave
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20120221/16302185/attachment.htm>