[MPICH2-dev] mpid/ch3/channel

[Brian R. Toonen]

Perhaps some definitions will help.  In MPICH2, a device is code that
implements the abstract device interface (ADI3).  Device implementation code
is placed in a subdirectory under src/mpid.

The CH3 device, located in src/mpid/ch3, implements the ADI3 in terms of
another interface that we call the CH3 (or channel) interface.  The CH3
interface is significantly smaller than the ADI3 and requires the programmer
to know substantially less about internal MPICH2 data structures.  A channel
is code that implements the CH3 interface.  Channel implementation code is
placed in subdirectory under src/mpid/ch3/channels.  The shared memory (or
shm) channel is one example of a channel and is located in
src/mpid/ch3/channel/shm.  Other channels include sockets (sock), Infiniband
(ib), and sockets and shared memory (ssm).

So, if you are interested in developing a device from scratch, implementing
all of the ADI3 routines yourself, then you should add a device to
src/mpid/<DEVICE>.  However, if you want to take advantage of the existing
CH3 device with its smaller and simpler CH3 interface, then you would add a
new channel to src/mpid/ch3/channels/<CHANNEL>.

To be fair, the CH3 interface is simpler than the ADI3; however, the ADI3
provides the most flexibility.  This flexibility can be important when
squeezing out those last few nanoseconds in order to obtain maximum possible
performance.  However, we recommend starting with a channel and then moving
to a separate device only if performance necessitates it.

--brian


> -----Original Message-----
> From: owner-mpich2-dev at mcs.anl.gov 
> [mailto:owner-mpich2-dev at mcs.anl.gov] On Behalf Of Noah Yan
> Sent: Friday, December 12, 2003 10:47
> To: mpich2-dev at mcs.anl.gov
> Subject: [MPICH2-dev] mpid/ch3/channel
> 
> 
> Dear All,
> 
> I am new to MPICH2 development. And currently I am involving 
> develop a 
> new device implementation based on MPICH2 ADI3. And I am 
> looking at the 
> source code and implementation doc for it.
> 
> One questions: src/mpid/ch3 is for ADI3 device 
> implementation, what is 
> the channel dir for in /mpid/ch3/channel, they are folder 
> like ib, shm, 
> sock?? If I understand ADI and channel correclty, they are differenct 
> devices for ADI3, like sock, shm, etc??
> 
> If I want to have another channel under ch3, should I 
> introduce it under 
> channel dir or directly under mpid/??
> 
> Thanks very much.
> 
> Noah Yan
>