[mpich2-dev] question about communication optimization on shared memory

[Darius Buntinas]

MPICH configure will try to use posix shared memory (i.e., mmapping a file), and if that fails it will try to use sysv shared memory (i.e., shmat).  If that fails, I think configure aborts (but it may just disable shared memory, I'm not sure).  Sysv shared memory typically has tight limits on the amount of shared memory that can be allocated per process and per node, so you don't really want to use that.

At init time, the processes query the process manager for the topology (i.e., which processes are on which nodes), and based on that decide which pairs of processes should communicate using shared memory and which should use the network.

Currently collectives are implemented using point-to-point messages, so some messages will go over shared memory and some over the network.  The communication patterns used by the collectives are optimized to take the topology into account.  MPICH does have an interface to allow implementors to override the default collective algorithms, such as using shared memory, but we haven't done this yet.

-d


On Oct 14, 2012, at 8:31 PM, Shigang Li wrote:

> Dear Sir or Madam,
> 
> I'm a student from UIUC, and have some questions about communication optimizations on shared memory in state-of-art mpich2.
> 
> 1) When configure mpich2, does it support shared memory optimization in default? If so, can it select the communication pattern automatically according to hardware architecture? For example in a SMPs cluster, using shared memory optimization within a node and using socket cross nodes?
> 
> 2) What about collective communication on shared memory? Is it implemented on top of the optimized point to point communication or use other mechanisms?
> 
> Best Regards,
> Shigang.