[MPICH] Using non-default ethernet interfaces

[Jeff Squyres]

Thanks Ralph; that also worked (I used Anthony's solution before  
reading yours).  Your solution allowed mpdboot to use my alternate  
hostnames without barfing.

I had originally tried having a hostfile like this:

m1$ cat hostfile
m1-fast
m2-fast
m1$ mpdboot -n 2 -f hostfile

And got a weird error (see my prior e-mail).  Shouldn't mpdboot be  
smart enough to be able to figure out that it is on the same host as  
m1-fast, and not require the --ifhn argument on the command line?   
Based on the behavior of prior versions of MPICH (and LAM/MPI), I had  
expected that simply supplying the -fast hostnames in the hostfile a)  
would allow mpdboot to work properly, and b) MPI apps started via the  
mpd would "inherit" the -fast hostnames and use them for MPI pt2pt  
communication.

This is somewhat of a moot point since your method works; I mention  
it simply as a usability / Law of Least Astonishment issue.

(BTW, messages sent through this listserver seem to take quite a  
while to be delivered; 30+ minutes sometimes...)



On May 16, 2007, at 6:29 PM, Ralph Butler wrote:

> This can be a bit tricky to get right no matter which way you try  
> to do it, and there are
> multiple ways to pull it off.  You can set the interface when mpds  
> are started and then
> the user pgms will use that same interface.  Or, you can let the  
> mpds use one interface
> and then have your program use a different interface at runtime;  
> this requires using
> the -machinefile option on mpiexec.
>
> I will assume for the present that you want to set the fast  
> interface for the mpds and
> then have the pgm inherit that interface.  From your email, I  
> assume you are also
> planning to start the mpds via mpdboot; that is actually the  
> goofiest part.  Here's how
> it goes.
>
> Let's assume that some machines are named m1, m2, and m3.  Let's  
> also assume that
> each host has a second name associated with a fast interface, e.g.  
> m1-fast, m2-fast, and
> m3-fast.  Finally, we will assume that you are going to run mpdboot  
> from m1 and that you
> want to use the fast interfaces.  Then you might create a hosts  
> file named myhosts that
> contains:
>     m2-fast
>     m3-fast
> Since we are being picky about interfaces, I would leave m1/m1-fast  
> out of the file and handle
> it via the cmd-line.  It can get very confusing for mpdboot to know  
> which IP to associate with
> a given host.  Ultimately, it assumes that info about the local  
> host is reliable from the cmd-line.
> So, you do this:
>     mpdboot -v -f myhosts -n 3 --ifhn=m1-fast
> That --ifhn arg is the tricky part.  It is the ifhn to be  
> associated with the local host.  It will NOT be
> obtained from myhosts.  mpdboot will start one mpd on the local  
> machine (m1 using the ifhn
> m1-fast) and then will start 2 others on m2-fast and m3-fast.  Now,  
> mpich2 pgms should also
> use those interfaces.
>
> --ralph
>
> On WedMay 16, at Wed May 16 5:34PM, Jeff Squyres wrote:
>
>> Greetings.  I'm trying to run MVAPICH2 over ethernet to do some  
>> performance comparisons, but I'm having a heck of a time trying to  
>> figure out how to use a non-default TCP interface.
>>
>> Specifically, eth0 is my "normal" gigE network (the IP address  
>> associated with the hostname).  But I want to run an MVAPICH2 job  
>> over ib0 -- my IPoIB interface.
>>
>> I looked through the user documentation and didn't see anything  
>> about how to do this -- did I miss it?  Pointers would be greatly  
>> appreciated.
>>
>> Thanks.
>>
>> -- 
>> Jeff Squyres
>> Cisco Systems
>>


-- 
Jeff Squyres
Cisco Systems