[MPICH] Selecting which ethernet interface to use
jsquyres at cisco.com
Thu Aug 23 07:07:29 CDT 2007
On Aug 22, 2007, at 10:41 PM, Darius Buntinas wrote:
> I don't know much about slurm, but looking at the docs, I bet it
> doesn't support that.
> But if you can set environment variables for each process in slurm,
> you can set the address other processes will use to connect to a
> process by setting the MPICH_INTERFACE_HOSTNAME environment
> variable for that process.
> E.g., if the address for a node is bb01 and the address for the ib
> interface is bb01-ib, set MPICH_INTERFACE_HOSTNAME=bb01-ib for any
> processes on that node.
Hmm. Is there a way to *not* specify the hostname? SLURM can
distribute environment variables to the launched processes, but it
usually sends the same values to all processes. Eg:
setenv FOO bar
srun -N 4 env | grep FOO
(where each of those came from a different node)
So it would be pretty messy to setup a MPICH_INTERFACE_HOSTNAME
specific for each host. The desired ethernet interface is the same
on all of my hosts (ib0); is there a way to tell all MPICH2 processes
to use ib0?
FWIW: I tried setting MPICH_INTERFACE_HOSTNAME to "ib0" and that
didn't work; for the heckuvit I also tried setting MPICH_INTERFACE to
"ib0" and that didn't work either (on the long shot that
MPICH_INTERFACE was a host-unspecific variant of
Also, it seems a little odd that you specify "<hostname>-<interface>"
when the name of the variable is MPICH_INTERFACE_HOSTNAME --
shouldn't it be MPICH_HOSTNAME_INTERFACE to match the ordering? Just
a nit. :-)
> Another way, if you can use mpd, would be to create a machinefile
> that looks like:
I don't really want to use mpd -- kinda the point of SLURM exporting
a PMI interface is to avoid using mpd and directly launch MPI
processes through the SLURM interface itself (i.e., I don't have to
write a script -- I can just srun my MPI processes directly).
Plus, if I used mpd, I'd have to glean the hosts that were allocated
to me from SLURM to create a hostfile, then make the translations in
that hostfile for what the corresponding public ethernet interface
name is in the ifhn clause, etc. It would be much simpler if I could
just setenv a variable that says what ethernet interface to use on
Am I stuck? Do I need to go this route (use mpd/create a hostfile)
to use something other than eth0?
More information about the mpich-discuss