<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=utf-8">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<title>RE: [mpich-discuss] mpich2 and IPoIB</title>
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EstiloDeEmail18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page Section1
{size:612.0pt 792.0pt;
margin:70.85pt 3.0cm 70.85pt 3.0cm;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=PT-BR link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Hi,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>We got something here, now the communications goes through
infiniband. The machinefile for smpd is a bit different from the mpd.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Also tested the machinefile with 1.0.7 and works too.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Fabio<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Jayesh Krishna
[mailto:jayesh@mcs.anl.gov] <br>
<b>Sent:</b> Friday, October 31, 2008 2:03 PM<br>
<b>To:</b> 'Fabio Kenji'<br>
<b>Cc:</b> mpich-discuss@mcs.anl.gov<br>
<b>Subject:</b> RE: [mpich-discuss] mpich2 and IPoIB<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p><span style='font-size:10.0pt'> Hi,<br>
Can you try specifying hostnames and interface ipaddresses in your
machinefile (also see the difference in the way interface ipaddr is specified)
?<br>
<br>
==========================================<br>
Hostname_of_10_2_5_10 -ifhn 10.2.10.10<br>
Hostname_of_10_2_5_11 -ifhn 10.2.10.11<br>
==========================================<br>
<br>
Regards,<br>
Jayesh<br>
<br>
-----Original Message-----<br>
From: Fabio Kenji [<a href="mailto:kenji_japanese@ig.com.br">mailto:kenji_japanese@ig.com.br</a>]<br>
Sent: Friday, October 31, 2008 6:39 AM<br>
To: Jayesh Krishna<br>
Cc: mpich-discuss@mcs.anl.gov<br>
Subject: Re: [mpich-discuss] mpich2 and IPoIB<br>
<br>
Hi,<br>
<br>
Compiled the 1.0.8 with the following options:<br>
./configure --enable-cxx --with-pm=smpd --prefix=<path> make &&
make install<br>
<br>
to started the smpd using<br>
<path>smpd -s<br>
<br>
to launch the program I'm using<br>
<path>mpiexec -n 4 -machinefile mf <program><br>
<br>
the mf file contains:<br>
10.2.5.10 ifhn=10.2.10.10<br>
10.2.5.11 ifhn=10.2.10.11<br>
<br>
where 10.2.5.xx is ethernet and 10.2.10.xx is infiniband<br>
<br>
and the communications are going through ethernet<br>
<br>
Fabio<br>
<br>
2008/10/30, Jayesh Krishna <jayesh@mcs.anl.gov>:<br>
><br>
><br>
><br>
> Hi,<br>
> Can you try the latest stable release (1.0.8) of MPICH2
?<br>
> I just tried entering a non-existant ipaddress in the
machinefile using the "-ifhn" option and my program failed (expected
failure). A quick look of the smpd debug output shows that smpd is setting the
right (as per the "-ifhn" args) environment before launching the
process.<br>
> Please let us know if MPICH2 1.0.8 + smpd +
"-ifhn" option does not work for you.<br>
><br>
> Regards,<br>
> Jayesh<br>
><br>
> -----Original Message-----<br>
> From: Fabio Motezuki [<a href="mailto:kenji_japanese@ig.com.br">mailto:kenji_japanese@ig.com.br</a>]<br>
> Sent: Wednesday, October 29, 2008 8:36 PM<br>
> To: Jayesh Krishna<br>
> Cc: mpich-discuss@mcs.anl.gov<br>
> Subject: Re: [mpich-discuss] mpich2 and IPoIB<br>
><br>
> Returning, I compiled de 1.0.8rc1 source and using mpd the ifhn in
the mpd.hosts file worked well, but not with interface name resolution, just
using ip directly.<br>
><br>
> In 1.0.7 I tried to use ifhn in the machinefile for smpd execution,
but no effect even when using the ip address of the infiniband interface.<br>
><br>
><br>
> Fabio<br>
><br>
><br>
><br>
> Jayesh Krishna escreveu:<br>
><br>
> ><br>
> > Hi,<br>
> > Yes, you can use the "-ifhn" option in
the machinefile with smpd.<br>
> > Let us know if you have any problems.<br>
> ><br>
> > Regards,<br>
> > Jayesh<br>
> ><br>
> > -----Original Message-----<br>
> > From: owner-mpich-discuss@mcs.anl.gov ><br>
> [<a href="mailto:owner-mpich-discuss@mcs.anl.gov">mailto:owner-mpich-discuss@mcs.anl.gov</a>]
On Behalf Of Rajeev Thakur ><br>
> Sent: Monday, October 27, 2008 2:22 PM > To:<br>
> mpich-discuss@mcs.anl.gov > Subject: RE: [mpich-discuss] mpich2
and<br>
> IPoIB > > Probably :-). From doing a grep in the smpd
directory it<br>
> looks like it > might accept "-ifhn hostname"<br>
> ><br>
> > Rajeev<br>
> ><br>
> > > -----Original Message-----<br>
> > > From: owner-mpich-discuss@mcs.anl.gov > ><br>
> [<a href="mailto:owner-mpich-discuss@mcs.anl.gov">mailto:owner-mpich-discuss@mcs.anl.gov</a>]
On Behalf Of Fabio Motezuki <br>
> > > Sent: Monday, October 27, 2008 2:12 PM > > To:<br>
> mpich-discuss@mcs.anl.gov > > Subject: Re: [mpich-discuss]
mpich2 and<br>
> IPoIB > > > > Thanks Rajeev, > >
> > I'll try it, perhaps is there<br>
> any options like this for smpd?<br>
> > ><br>
> > > Fabio<br>
> > ><br>
> > ><br>
> > > Rajeev Thakur escreveu:<br>
> > > > Try specifying the interface name using ifhn= in
the > ><br>
> mpd.hosts file as > > > described in Sec 5.1.5 of the
installation<br>
> guide.<br>
> > > ><br>
> > > <a
href="http://www.mcs.anl.gov/research/projects/mpich2/documentation/">http://www.mcs.anl.gov/research/projects/mpich2/documentation/</a><br>
> > > files/mpich2-1<br>
> > > > .0.8-installguide.pdf<br>
> > > ><br>
> > > > Rajeev<br>
> > > ><br>
> > > ><br>
> > > >> -----Original Message-----<br>
> > > >> From: owner-mpich-discuss@mcs.anl.gov >
> >><br>
> [<a href="mailto:owner-mpich-discuss@mcs.anl.gov">mailto:owner-mpich-discuss@mcs.anl.gov</a>]
On Behalf Of > > Fabio<br>
> Motezuki > > >> Sent: Monday, October 27, 2008 5:35
AM > > >> To:<br>
> mpich-discuss@mcs.anl.gov > > >> Subject:
[mpich-discuss] mpich2 and<br>
> IPoIB > > >> > > >> Hi all, >
> >> > > >> I'm working on a cluster<br>
> where each node is connected with two > > >> networks:<br>
> > > >><br>
> > > >> eth0 - gigabit ethernet<br>
> > > >> ib0 - IP over infiniband<br>
> > > >><br>
> > > >> I would like to use the ib0 net for all mpi
communications,<br>
> but > > >> when I start the example cpi all
comunications go through<br>
> eth0 is > > this the > > >> expected
behavior?<br>
> > > >><br>
> > > >> I'm launching the example program with
"mpiexec > ><br>
> -machinefile mf -n 16 > > >> ./cpi" where the file
"mf" contains the<br>
> ip address of infiniband > > >> cards.<br>
> > > >><br>
> > > >> Fabio<br>
> > > >><br>
> > > >><br>
> > > >><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > ><br>
> > ><br>
> ><br>
></span><o:p></o:p></p>
</div>
</div>
</body>
</html>