[MPICH] Can I ask MPI to not copy the data to shared-mem ?

[Darius Buntinas]

Hmm.  I can't see a way that an MPI implementation could (efficiently 
and reliably) check that two virtual addresses from different processes 
point to the same location in a shared memory region.  An MPI 
implementation wouldn't have enough information to do what you're asking.

Rusty Lusk mentioned to me that there was a message-passing library 
called p4 that would allow this kind of thing.  You would allocate a 
message using p4_msg_alloc(), then fill the message and send it.  The 
receiver can receive the message without specifying a destination 
buffer.  In this case, on a shared memory machine, the receiver gets a 
pointer to the original buffer allocated by the sender using 
p4_msg_alloc(), and no copy is made.

-d

On 10/09/2007 11:53 AM, chong tan wrote:
> Well, shared-mem and sending 0 byte does not work if the processes are 
> on differnt boxes.  My thought is that since MPICH already make the 
> dicission on wheter to use shared memory.  It is a nature fit that the 
> 'no copy' rule be applied in that domain.
>  
> thanks
> tan
> 
> 
> */Darius Buntinas <buntinas at mcs.anl.gov>/* wrote:
> 
> 
> 
>     On 10/08/2007 01:38 PM, chong tan wrote:
>      > I am interested in the case that the send and recieve buffer are the
>      > same buffer allocated from shared-memory, in which case there is
>     no need
>      > for the data copying, send and recieve just only do the sync.
> 
>     In this case, why not use a zero-byte message? The sender fills in the
>     shared buffer, does a write barrier, then calls MPI_Send with a
>     zero-byte message. The receiver calls MPI_Recv, to receive zero bytes,
>     then when it returns, it does a read barrier and reads the data from
>     the
>     shared buffer.
> 
>      > A side question, when MPI_Send is called, does MPICH copy the
>     data into
>      > a intermediate global buffer ? and have the data copy from this same
>      > buffer on MPI_Recv call ?
> 
>     It depends on the channel and what you're sending. For non-contiguous
>     data, the data may be packed into a contiguous buffer before sending
>     and
>     unpacked from a contiguous buffer when receiving. Otherwise, if a
>     channel uses sockets, an intermediate buffer is not used by MPICH2, but
>     the OS will perform intermediate copies. For a channel that uses a
>     high-performance network, an intermediate buffer is used for small
>     messages, but for large messages the data is transferred directly from
>     or into the user's buffer.
> 
>     -d
> 
>      > tan
>      >
>      >
>      > */Darius Buntinas /* wrote:
>      >
>      >
>      > Well, you specify one buffer in the call to MPI_Send and another
>     buffer
>      > in MPI_Recv, so there would have to be at least one copy. You
>     can't do
>      > it without any copies (otherwise, how would the data get from the
>     send
>      > buffer to the receive buffer?).
>      >
>      > MPICH2 does support communication over shared memory (as opposed to
>      > over
>      > a network) which improves performance for intranode communication.
>      > Configure with --with-device=ch3:ssm or --with-device=ch3:nemesis.
>      >
>      > Darius
>      >
>      >
>      > On 10/08/2007 12:22 PM, chong tan wrote:
>      > > If I the pointer passed to MPI_Send and MPI_Recieve are already
>      > shared
>      > > memory, and need not be copied at all ?
>      > >
>      > > If so, How ?
>      > >
>      > > thanks
>      > > tan
>      > >
>      > >
>      > >
>      >
>     ------------------------------------------------------------------------
>      > > Be a better Heartthrob. Get better relationship answers
>      > > from
>      > > someone who knows.
>      > > Yahoo! Answers - Check it out.
>      >
>      >
>      >
>     ------------------------------------------------------------------------
>      > Need a vacation? Get great deals to amazing places
>      > on
>      > Yahoo! Travel.
> 
> 
> ------------------------------------------------------------------------
> Boardwalk for $500? In 2007? Ha!
> Play Monopoly Here and Now 
> <http://us.rd.yahoo.com/evt=48223/*http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow> 
> (it's updated for today's economy) at Yahoo! Games.