[mpich-discuss] MPI_Allreduce increases the performance of MPI_Alltoallv?

[Dave Goodell]

On Jul 3, 2012, at 3:28 PM CDT, Dave Goodell wrote:

> On Jul 3, 2012, at 1:39 PM CDT, Li, Lihua (UMSL-Student) wrote:
> 
>> Dear MPICH users,
>> 
>> Do any one happen to have such an experience, when MPI_Allreduce "seems to" make MPI_Alltoallv faster?
> […]
>> The Jumpshot image for the modified version has greater variation of MPI_Alltoallv time among processes, while there is no MPI_Allreduce operations. It gives me a weird feeling that MPI_Allreduce operations are "regulating" the behavior of MPI_Alltoallv.
>> 
>> Anyone got hint of what's going on? 
> 
> The allreduce operations have barrier semantics because of the data dependencies.  It could be that these "barriers" are reducing the number of unexpected messages inside the alltoallv algorithm, which is causing the performance improvement.

To add to this, one thing you might try to do is set the environment variable "MPICH_ALLTOALL_THROTTLE" to different values.  Its default value is 4.  Setting it to 0 causes all irecvs/isends to be posted at once.

What version of MPICH2 are you using? (hopefully something modern)

-Dave