[mpich2-dev] question about communication optimization on shared memory

[Darius Buntinas]

You can control it with --with-shared-memory[=auto|sysv|mmap].  You can check which is being used by looking at the configure output (look for "Using a memory-mapped file for shared memory" or "Using SYSV shared memory").

-d

On Oct 16, 2012, at 3:11 PM, Shigang Li wrote:

> Hi,Darius,
> Thank you and this really makes sense. I still have one question. Is it possible for a MPICH programmer to know which configuration is using, e.g. posix shared memory,  sys V shared memory or disabled?
> 
> Best Regards,
> Shigang.
> 
> On Mon, Oct 15, 2012 at 10:18 AM, Darius Buntinas <buntinas at mcs.anl.gov> wrote:
> 
> MPICH configure will try to use posix shared memory (i.e., mmapping a file), and if that fails it will try to use sysv shared memory (i.e., shmat).  If that fails, I think configure aborts (but it may just disable shared memory, I'm not sure).  Sysv shared memory typically has tight limits on the amount of shared memory that can be allocated per process and per node, so you don't really want to use that.
> 
> At init time, the processes query the process manager for the topology (i.e., which processes are on which nodes), and based on that decide which pairs of processes should communicate using shared memory and which should use the network.
> 
> Currently collectives are implemented using point-to-point messages, so some messages will go over shared memory and some over the network.  The communication patterns used by the collectives are optimized to take the topology into account.  MPICH does have an interface to allow implementors to override the default collective algorithms, such as using shared memory, but we haven't done this yet.
> 
> -d
> 
> 
> On Oct 14, 2012, at 8:31 PM, Shigang Li wrote:
> 
> > Dear Sir or Madam,
> >
> > I'm a student from UIUC, and have some questions about communication optimizations on shared memory in state-of-art mpich2.
> >
> > 1) When configure mpich2, does it support shared memory optimization in default? If so, can it select the communication pattern automatically according to hardware architecture? For example in a SMPs cluster, using shared memory optimization within a node and using socket cross nodes?
> >
> > 2) What about collective communication on shared memory? Is it implemented on top of the optimized point to point communication or use other mechanisms?
> >
> > Best Regards,
> > Shigang.
> 
>