[mpich-discuss] best/fastest way to get a node communicator

[Rhys Ulerich]

> On 01/17/2012 01:24 PM, Jeff Hammond wrote:
>> I'm interested in being able to create a communicator for each node.
>> I have custom APIs for this on Blue Gene and Cray, but that doesn't
>> help on clusters.
>>
>> I was thinking of doing the following:
>> - call gethostname on each rank
>> - gather these values to root
>> - sort the array and assign a different color number for each unique
>> value in a new array
>> - scatter the color array and call comm_split
>>
>> Does anyone know of a better/faster way to do this?

This may avoid the memory overhead on root and avoids the sort.

- call MPI_Get_processor_name as Jim Dinan suggested
- hash the processor name into a nonnegative int color (with care as
to the chosen hash function)
- MPI_Comm_split on the color
- Set value '1' into some integer buffer on all ranks.
- MPI_Allreduce MPI_SUM on each integer buffer to find the number of
ranks in each rank's color-communicator
- MPI_Allreduce to find the minimum and maximum of the ranks in each
color-communicator across COMM_WORLD
- If minimum and maximum are not identical, throw away the bad
node-specific communicators (you encountered a hash collision), add
some salt to the hash (maybe an iteration number), and repeat the
process.

Presumably you won't be hit by hash collisions indefinitely.  With a
good hash function, you probably won't hit any collisions at all.

It is likely not faster (as it is a bit chatty) but I've never
measured it.  Nor implemented it.

- Rhys