[mpich-discuss] Hydra process placement verification

[Jeffrey J. Evans]

I am trying to learn how to correctly verify hydra process placement. 

For example: I have an MPI program that uses 6 processes. On a node with 2 quad-core processors I setup the following host file

hpn10:8 binding=user:1,2,3,5,6,7 

My MPI program grabs the process information from each process using MPI_Get_processor_name()

The resulting output:
# 000: OK on hpn10/0, EA: 1, rank: 0 ptrn 1 tick: 1.000000e-06
# 001: OK on hpn10/1, EA: 1, rank: 1 ptrn 1 tick: 1.000000e-06
# 002: OK on hpn10/2, EA: 1, rank: 2 ptrn 1 tick: 1.000000e-06
# 003: OK on hpn10/3, EA: 1, rank: 3 ptrn 1 tick: 1.000000e-06
# 004: OK on hpn10/4, EA: 1, rank: 4 ptrn 1 tick: 1.000000e-06
# 005: OK on hpn10/5, EA: 1, rank: 5 ptrn 1 tick: 1.000000e-06

I was expecting to see something like:
# 000: OK on hpn10/1, EA: 1, rank: 0 ptrn 1 tick: 1.000000e-06
# 001: OK on hpn10/2, EA: 1, rank: 1 ptrn 1 tick: 1.000000e-06
# 002: OK on hpn10/3, EA: 1, rank: 2 ptrn 1 tick: 1.000000e-06
# 003: OK on hpn10/5, EA: 1, rank: 3 ptrn 1 tick: 1.000000e-06
# 004: OK on hpn10/6, EA: 1, rank: 4 ptrn 1 tick: 1.000000e-06
# 005: OK on hpn10/7, EA: 1, rank: 5 ptrn 1 tick: 1.000000e-06 

My real problem is that I cannot locate in the documentation how hydra uses hwloc to bind processes to cores. Does hydra need paths to hwloc bin and lib directories? Does mpich2 need to be rebuilt with configuration information regarding the location of hwloc binaries and libraries? 

To provide more info - my mpich2-1.1.1p1 build was done with the pm:hydra configuration set, but hwloc was installed later. Does mpich2 need to be rebuilt? 
The resource manager = Torque, scheduler = Maui.

How can I verify core placement? 

Jeffrey J. Evans
jje at purdue.edu
http://web.ics.purdue.edu/~evans6/