itaps-parallel entities arg. in iMeshP_push... on non-owner side?

[Devine, Karen D]

Onkar  (and Jason):

Yes, I still had your Oct 6 email in my mailbox.  Thanks for bringing it
back to my attention.

Here's the short answer; more detailed comments are below.

For SC08, we should probably demonstrate just the blocking tag exchange
iMeshP_pushTags*().  The more I think about the non-blocking versions, the
more confused I get, so we should probably take our time and really work out
the details before presenting those functions.  You make some good
suggestions.  Perhaps -- after SC08 -- we can write an outline of exactly
how they should be used.

I still don't understand the destination tag argument; perhaps Jason can
help me on that.  What is it, exactly?  Is it a remote tag handle?  If so,
couldn't it be different on each of the receiving processors (so having one
in the call would be insufficient)?  Or is it a way just to specify a
different tag name on the receiving processors (in which case the
documentation is wrong)?  Jason, would you please give an example of a
source and destination tag so I can get the documentation right?  Thanks!

There are a few more comments below.

On 10/6/08 4:06 PM, "Onkar Sahni" <osahni at scorec.rpi.edu> wrote:

>> IMeshP_pushTagsEnt is a synchronous function, so all processors would call
>> it.  Each part would list only the entities it wanted to push.  Parts with
>> no entities to push would still call the function, but with no input
>> entities.
>>
>> IMeshP_iPushTagsEnt is an asynchronous function, so Carl's answer below
>> applies.  Only P0 would call the function; the function would return a
>> request handle.  P0 would later have to check the requests to determine
>> whether the request completed (iMeshP_Wait).  P1 would have to poll for
>> requests and perform the update as Carl describes.
>
>   Overall it makes sense but I still have some concerns, see below:
>
> (0) I am assuming synchronous for iMeshP_pushTagsEnt() doesn't imply usage
> of MPI_Ssend() (on sender side) or any specific mode of send at
> implementation level (one can say both synchronous or blocking in case of
> iMeshP_pushTagsEnt() means one and the same thing at iMeshP-level to a
> user).

Yes, when we say "synchronous" or "blocking," we are not specifying which
MPI calls must be used.  Rather, we are saying that the sender and receiver
must call the function at roughly the same time.

>
> (1) Does iMeshP_pushTagsEnt() need to be called by "all processes" (Karen
> I just want to make it clear as you mentioned "all processors would call
> it") or can it be called for a "segment" of shared entities only on parts
> sharing that segment?

I don't see a reason why it couldn't be called for a segment of shared
entities, as long as both the owners and those processes with copies of the
segment both called it.  Does that make it more difficult to implement?

>
> (2) iMeshP_iPushTagsEnt() implies iMeshP_RequestHandle(s) in
> iMeshP_poll...() can be output but then why we call them "Request"
> (considering MPI has MPI_Request and MPI_Status).

Are you suggesting that iMeshP_iPushTags*() should be push-and-forget and,
therefore, not return a request handle?

>
> I am also hoping that we can be close to MPI paradigm so that it will be
> "easy-to-use" for users/applications (and also implementation can comply
> to specification) so I have other related questions:
>
> before questions this is the map between iMeshP_ and MPI_ I have in my mind:
> iMesh_pushTagEnt...() ==> uses MPI_?send() (on sending side)
> iMesh_pushTagEnt...() ==> uses MPI_?recv() (on receiving side)
> iMeshP_Wait...()      ==> uses MPI_wait() (for a given request)
> iMeshP_poll...()      ==> uses MPI_probe() or MPI_Iprobe()
>                           (on receiving side, which uses MPI_Status)
>
> (where "?" implies any mode for send/recv. as per choice within
> implementation like MPI_Ssend or  MPI_Send or MPI_Isend or MPI_Recv or
> MPI_IRecv)
>
> Here are further questions:
>
> (a) Is it an error to call iMeshP_iPushTagsEnt() (and may be also in
> iMeshP_pushTagsEnt()) with receivers in "entities" array (i.e., non-owners
> in case of push) as in case of iMeshP_iPushTagsEnt() one can avoid call to
> iMeshP_poll...() if receivers are provided in "entities" list?
>
> (b) What happens in case of iMeshP_pullT..() and iMesh_iPullT...() (as per
> my understanding non-owners sends and owner receives)? So
> iMeshP_*pull...() is called with non-owners in "entities" list (as
> senders/owners filled the "entities" list in the case of
> iMesh_*push...())?
>
> (c) When will poll for messages end, I recall we discussed before that it
> would involve sync./collective stage. If iMeshP_poll...() leads to having
> a sync. stage then user should have the flexibility to provide "entities"
> list on receiving side in iMeshP_iPushTagsEnt() or iMeshP_iPullTagsEnt()
> to avoid call to iMeshP_poll...()?
>
> There might be more questions as we implement more on these functions ;)...
>
> Thanks,
> Onkar
>
>>
>> Karen
>>
>>
>> On 10/4/08 8:39 PM, "Carl Ollivier-Gooch" <cfog at mech.ubc.ca> wrote:
>>
>>> Onkar Sahni wrote:
>>>> Karen,
>>>>
>>>> I want to make sure that I understand, so I will ask this question in a
>>>> slightly different way --- what if I want to exchange tag-data for only
>>>> one entity (e0) which has copies in two parts P0 and P1, with P0 as
>>>> owner,
>>>> so I understand that I call iMeshP_push... from P0 (which sends) but do
>>>> I
>>>> make iMeshP_push... call on P1 or not? If so then is entities arg.
>>>> empty
>>>> and if not, then when is the counter-receive of point-to-point
>>>> communication posted on non-owner side?
>>>
>>> This is one of the things the pollForRequests does: identify incoming,
>>> unsolicited messages and do sensible things with them.  Naturally, the
>>> contents of the message will have to provide some info about what
>>> "sensible" implies, because pollForRequests will have to handle not only
>>> receiving pushed tags, but also various parts of micro-migration.
>>> Details will of course be implementation dependent, but pollForRequests
>>> is arguably the most complex function in the interface to implement.
>>>
>>> Carl
>>>
>>> --
>>> ------------------------------------------------------------------------
>>> Dr. Carl Ollivier-Gooch, P.Eng.                   Voice: +1-604-822-1854
>>> Associate Professor                                 Fax: +1-604-822-2403
>>> Department of Mechanical Engineering             email: cfog at mech.ubc.ca
>>> University of British Columbia              http://www.mech.ubc.ca/~cfog
>>> Vancouver, BC  V6T 1Z4                  http://tetra.mech.ubc.ca/ANSLab/
>>> ------------------------------------------------------------------------
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>
>