<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">Hello,</span></font><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">thank you very much for your answer</span></font></div></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><div>Le 12 déc. 2011 à 16:39, Pavan Balaji a écrit :</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><blockquote type="cite"><br class="Apple-interchange-newline">2/ with Hydra (1.4.1p1) , is it possible to use 10 GigE TCP interface<br></blockquote><blockquote type="cite">We have machines with two TCP interfaces (eth0 = 1Gb/s, eth2 = 10Gb/s)<br></blockquote><blockquote type="cite">with Hydra I can't use eth2 even when specifying -iface eth2<br></blockquote><br>This should work correctly. Just to make sure you are using this right: setting the -iface option makes sure that your application is using the eth2 interface, so MPI communication will use eth2. Hydra itself will use the default network interface for launching processes, unless it finds a local interface in your host file list. If the SGE provided host list is using the 1GE interface, it will use 1GE.<br><br>You might or might not care about what network interface is used for launching processes, but I want to make sure you are checking the network usage on the right machines.<br></span></blockquote></div><div><br></div><div><span class="Apple-style-span" style="font-size: 14px; ">Not sure to really understand</span></div><div><div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">Do you means that the [optional] machine file (mpiexec -f machines_file) will only be used to launch process ?</span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">To be clear, I want to be able to get with hydra + Grid Engine the following behavior :</span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">Manual test with a piece of code, between the machines ccwpge0061 | ccwpge0062 (</span></font><span class="Apple-style-span" style="font-size: 14px; ">ccwpge0061<b>p</b> | ccwpge0062<b>p</b> for secondary interface)</span></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">and using old mpd solution</span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">### on ccwpge0061 ### :</span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>>hostname; mpd --ifhn=ccwpge0061p --daemon --echo</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>ccwpge0061</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>mpd_port=50712</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">### on ccwpge0062 ### :</span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>>hostname; mpd -h ccwpge0061 -p 50712 --ifhn=ccwpge0062p --daemon --echo</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>ccwpge0062</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>mpd_port=56112</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>>mpdtrace -l</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>ccwpge0062_56112 (10.158.175.62)</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>ccwpge0061_50712 (10.158.175.61)</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i><br></i></span></font></div><div><span class="Apple-style-span" style="font-size: 14px; ">### Back on ccwpge0061 ###</span></div><div><span class="Apple-style-span" style="font-size: 14px; "><i>>mpdtrace -l</i></span></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>ccwpge0061_50712 (10.158.175.61)</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>ccwpge0062_56112 (10.158.175.62)</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i><br></i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>> mpiexec -machinefile /tmp/machines.eth2 -n 2 bin/advance_test</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i><br></i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>> dstat -n -N eth0,eth2</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>--net/eth0- --net/eth2-</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i> recv send: recv send</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i> 0 0 : 0 0 </i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i> 17k 42k:2065k 1158M</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>2209B 72k:2067k 1158M</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>1610B 24k:2063k 1158M</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>1738B 24k:2061k 1158M</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><i>2114B 24k:2064k 1158M</i></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div></div><div><span class="Apple-style-span" style="font-size: 14px; ">Ok, I use eth2 at </span><span class="Apple-style-span" style="font-size: 14px; ">10Gige/s </span><span class="Apple-style-span" style="font-size: 14px; "> (see ~1GB/s usage)</span></div></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">The same code (</span></font><i>advance_test</i>) run with hydra thru Gridengine :</div><div> <span class="Apple-style-span" style="font-size: 13px; ">mpiexec -rmk sge -iface eth2 -n $NSLOTS ./bin/advance_test</span></div><div><span class="Apple-style-span" style="font-size: 13px; "><br></span></div><div><span class="Apple-style-span" style="font-size: 13px; ">don't use secondary interface, but the first one (eth0 - 1Gb/s, see ~118MB/s)</span></div><div><span class="Apple-style-span" style="font-size: 13px; ">>dstat -n -N eth0,eth2</span></div><div><span class="Apple-style-span" style="font-size: 13px; "><div style="font-size: 17px; "><font class="Apple-style-span" size="4" style="font-size: 21px; "><span class="Apple-style-span" style="font-size: 13px; "> --net/eth0- --net/eth2-</span></font></div><div style="font-size: 17px; "><font class="Apple-style-span" size="4" style="font-size: 21px; "><span class="Apple-style-span" style="font-size: 13px; "> recv send: recv send</span></font></div><div style="font-size: 17px; "><font class="Apple-style-span" size="4" style="font-size: 21px; "><span class="Apple-style-span" style="font-size: 13px; "> 0 0 : 0 0 </span></font></div><div style="font-size: 17px; "><font class="Apple-style-span" size="4" style="font-size: 21px; "><span class="Apple-style-span" style="font-size: 13px; "> 479k 118M: 432B 140B</span></font></div><div style="font-size: 17px; "><font class="Apple-style-span" size="4" style="font-size: 21px; "><span class="Apple-style-span" style="font-size: 13px; "> 476k 118M:1438B 420B</span></font></div><div style="font-size: 17px; "><font class="Apple-style-span" size="4" style="font-size: 21px; "><span class="Apple-style-span" style="font-size: 13px; "> 478k 118M: 0 0 </span></font></div></span></div></span></font></div><div><span class="Apple-style-span" style="font-size: 14px; "><br></span></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><div>So, where to specify the equivalent ifhn option of mpd ? Does the -iface option should be sufficient?</div><div><br></div><div><br></div><div><br></div><div>Best reagards</div></span></font></div></div><div><div><div><div><div><div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; ">---------------</span><span class="Apple-style-span" style="font-size: 14px; "><br></span><span class="Apple-style-span" style="font-size: 14px; ">Bernard CHAMBON</span><span class="Apple-style-span" style="font-size: 14px; "><br></span><span class="Apple-style-span" style="font-size: 14px; ">IN2P3 / CNRS</span><span class="Apple-style-span" style="font-size: 14px; "><br></span><span class="Apple-style-span" style="font-size: 14px; ">04 72 69 42 18</span><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div></div></div></div></div><div><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div></div></div><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Courier; font-size: 18px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div><div><div><div><div><span class="Apple-style-span" style="font-size: medium;"><br></span></div></div></div></div></div></span></div></body></html>