itaps-parallel Asynchronous functions in parallel interface

[Jason Kraftcheck]

Tim Tautges wrote:
> Conclusion:
> 
> After discussing after Lori hung up, we concluded that:
> 1) Asynchronous calls are still needed and useful for iMeshP, for
> various reasons.
> 2) We should document the fact that although some amount of work can be
> done after the asynchronous iMeshP call returns, there may be a
> non-negligible amount of work that needs to be completed synchronously
> (i.e. after the application calls a wait or test iMeshP function).
> 

The only sane way to avoid race conditions with the current API is to update
the data when the application calls the wait/poll/test method.  It should be
documented that the data is updated only for complete requests and only when
a wait/poll/test function is called for that completed request.  Given that,
it should be apparent to the application developer that any computational
cost related to assimilating the data from an asynchronous communication
will be incurred during the wait/poll/test call.

- jason


> - tim
> 
> 
> Onkar Sahni wrote:
>> I do not think this is true that "asynchronous calls are really
>> synchronous", that is irrespective of the (below mentioned) ways it is
>> handled either by OS-level parts of MPI or mesh-library-level threads (I
>> do not think any forking is necessary at iMesh level by implementations,
>> one may choose to do that way but many parallel systems do not allow
>> forking like IBM BG/L, Cray XT3 and I think its limited on IBM BG/P).
>>
>> What MPI does and how it interacts with OS should not dictate the way we
>> provide functionality.
>>
>> On Jason's proposal, I do not recall discussion on this (I may have
>> missed
>> it) and I do not really see a reason why two tags (source and
>> destination)
>> are needed.
>>
>> We can discuss more details on it via emails but before detailed email
>> discussions I would instead recommend to schedule a phone conference to
>> discuss these issues (if others think it is required).
>>
>> - Onkar
>>
>>> Hi all,
>>>    Thinking more about Jason's proposal lead me to a really important
>>> question about our parallel interface.  If we're proposing to have
>> asynchronous functions like iMeshP_migrateEntity which return a request
>> handle, with the application calling the test or wait functions to test
>> whether a request has finished, where is the code which is executing
>> that request running?  In the case of MPI, there is presumably some
>> interaction with the OS, which is acting autonomously to fill buffers,
>> transfer them to other processors, etc.  On large parallel systems
>> that's handled by the OS-level parts of MPI; on workstations or
>>> clusters, that's probably done by some other thread or process running
>> on each node (or asynchronous calls are really synchronous).  For imesh,
>> though, are any of the implementations able to do that kind of forking?
>>>   I know MOAB never will.  In that case, I think asynchronous calls to
>>> iMeshP don't make sense, or they only make sense if they're just
>> pass-through functions to MPI, in which case they're superfluous.  I
>> think it's down in the implementation where asynchronicity is used.  For
>> example, when exchanging tags of owned entities, that function can
>> initiate and wait for multiple asynchronous messages.  Comments?
>>> - tim
>>>
>>> -- 
>>> ================================================================ "You
>> will keep in perfect peace him whose mind is
>>>    steadfast, because he trusts in you."               Isaiah 26:3
>>>
>>>               Tim Tautges            Argonne National Laboratory
>>>           (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>> phone: (608) 263-8485      1500 Engineering Dr.
>>>             fax: (608) 263-4499      Madison, WI 53706
>>>
>>>
>>
>>
>>
>>
>>
> 


-- 
"A foolish consistency is the hobgoblin of little minds" - Ralph Waldo Emerson