[mpich-discuss] MPI and threaded communication

[Joseph Gonzalez]

Hi Again,

Thanks for addressing my earlier question concerning thread and
MPICH2.  As part of our multithreaded system we are creating a
callback based communicator with buffered send and receive queues.
This seems like an obvious goal however I am having trouble
terminating pending receives at the end of execution.

All in the same MPI Process (pseudo-code) :

Thread 1
while(alive) {
  msg <- MPI_Recv(Any_Origin, Any_type)
  enqueue msg to handlers
}


Thread 2
while(alive) {
  msg <- dequeue outbound message
  MPI_Send(msg, msg.dest, msg.type)
}

At some point an additional thread sets alive to false and would like
to force the blocking Send and Recv to return (maybe with an error).
How can I terminate the blocking sends and receives? I tried using
MPI_ISend and MPI_IRecv followed with a MPI_Wait and again neither
awake from the MPI_Wait when MPI_Cancel is invoked.  From the
specification it is clear how MPI_Cancel behaves before MPI_Wait is
invoked but not after.  We are introducing this abstraction because
our algorithm will dynamically determine which nodes it needs to
communicate with at runtime and it will be difficult to directly
interleave algorithm and communication events.

Thank you,
Joey



On Sun, Feb 1, 2009 at 6:15 PM, Joseph Gonzalez <jegonzal at cs.cmu.edu> wrote:
> Hi,
>
> We are building a threaded, event driven C++ wrapper for MPICH2.  We would
> like to be able to simultaneously send and receive messages from multiple
> threads in the same process.  If two threads simultaneous try to send to two
> different rank destinations (on some remote machine) will they both transmit
> simultaneously or is their an internal mutex that will prohibit this from
> happening?
>
> Thanks,
> Joey
>



-- 
Joseph E. Gonzalez
Machine Learning Department
Carnegie Mellon University