<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="font-size: 16px; ">I answer to my own mail …</span><div><br style="font-size: 16px; "><div style="font-size: 16px; "><div>Le 13 déc. 2011 à 09:20, Bernard Chambon a écrit :</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="font-size: 12px; "><div><font class="Apple-style-span" size="4" style="font-size: 20px; "><span class="Apple-style-span" style="font-size: 12px; "><div><font class="Apple-style-span" size="4" style="font-size: 20px; "><span class="Apple-style-span" style="font-size: 12px; ">The same code (</span></font><i>advance_test</i>) run with hydra thru Gridengine :</div><div> <span class="Apple-style-span" style="font-size: 11px; ">mpiexec -rmk sge -iface eth2 -n $NSLOTS ./bin/advance_test</span></div><div><span class="Apple-style-span" style="font-size: 11px; "><br></span></div><div><span class="Apple-style-span" style="font-size: 11px; ">don't use secondary interface, but the first one (eth0 - 1Gb/s, see ~118MB/s)</span></div><div><span class="Apple-style-span" style="font-size: 11px; ">>dstat -n -N eth0,eth2</span></div><div><span class="Apple-style-span" style="font-size: 11px; "><div style="font-size: 15px; "><font class="Apple-style-span" size="4" style="font-size: 19px; "><span class="Apple-style-span" style="font-size: 11px; "> --net/eth0- --net/eth2-</span></font></div><div style="font-size: 15px; "><font class="Apple-style-span" size="4" style="font-size: 19px; "><span class="Apple-style-span" style="font-size: 11px; "> recv send: recv send</span></font></div><div style="font-size: 15px; "><font class="Apple-style-span" size="4" style="font-size: 19px; "><span class="Apple-style-span" style="font-size: 11px; "> 0 0 : 0 0 </span></font></div><div style="font-size: 15px; "><font class="Apple-style-span" size="4" style="font-size: 19px; "><span class="Apple-style-span" style="font-size: 11px; "> 479k 118M: 432B 140B</span></font></div><div style="font-size: 15px; "><font class="Apple-style-span" size="4" style="font-size: 19px; "><span class="Apple-style-span" style="font-size: 11px; "> 476k 118M:1438B 420B</span></font></div><div style="font-size: 15px; "><font class="Apple-style-span" size="4" style="font-size: 19px; "><span class="Apple-style-span" style="font-size: 11px; "> 478k 118M: 0 0 </span></font></div></span></div></span></font></div><div><br></div><div><font class="Apple-style-span" size="4" style="font-size: 20px; "><span class="Apple-style-span" style="font-size: 12px; "><div>So, where to specify the equivalent ifhn option of mpd ? Does the -iface option should be sufficient?</div></span></font></div></span></span></blockquote></div><div style="font-size: 16px; "><br></div><div style="font-size: 16px; ">My failed situation was due to an older version of mpich2 used to compile my test code</div><div style="font-size: 16px; ">With the latest version (MPICH2 Version: 1.4.1p1), and after a code compilation, I got the -iface option working fine, cool! </div><div style="font-size: 16px; "><br></div><div style="font-size: 16px; ">It's interesting to notice that a got (after 3 try) different values according to gcc or icc (Intel) compiler</div><div style="font-size: 16px; "><br></div><div style="font-size: 16px; ">with Intel compiler, I almost reach the limit of the second interface (~ 1GB/s) </div><div style="font-size: 16px; "><div style="font-size: 12px; ">> mpicc -cc=/usr/local/intel/cce/10.1.022//bin/icc -O2 …</div></div><div style="font-size: 12px; "><br></div><div style="font-size: 16px; "><div><div style="font-size: 12px; "><font class="Apple-style-span" size="4"><span class="Apple-style-span" style="font-size: 16px;"><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px;"><div><div><div>> dstat -n -N eth0,eth2</div><div>--net/eth0- --net/eth2-</div><div> recv send: recv send</div><div> 0 0 : 0 0 </div><div> 735k 77k:5720k 928M</div><div> 70B 29k:2268k 956M</div><div> 292B 28k:3838k 946M</div><div> 70B 30k:2170k 905M</div><div> 134B 24k:5453k 876M</div><div><br></div></div><div><br></div></div></span></font></span></font></div></div><div><div style="font-size: 16px; ">with GNU compiler, I got around 700MB/s</div><div style="font-size: 16px; "></div></div><div style="font-size: 13px; "><br></div><div style="font-size: 13px; "><div>>mpicc -cc=/usr/bin/gcc -O2 …</div><div><br></div><div> >dstat -n -N eth0,eth2</div><div>--net/eth0- --net/eth2-</div><div> recv send: recv send</div><div> 0 0 : 0 0 </div><div>1435B 22k:1499k 715M</div><div> 326B 23k:1420k 656M</div><div> 578B 22k:1429k 686M</div><div> 198B 21k:1403k 666M</div></div><div><br></div><div>(With older release of mpich2, using gcc compiler, and mpd ring, I reach the second interface limit (1GB/s) </div></div><div style="font-size: 16px; "><br></div><div style="font-size: 16px; ">Thank you very much, hydra (+ Grid Engine) is nice !</div><div style="font-size: 16px; "><br></div><div style="font-size: 16px; ">Best regards</div><div><br></div></div><div><div style="font-size: 16px; ">
<div><div><div><div><div>---------------<br>Bernard CHAMBON<br>IN2P3 / CNRS<br>04 72 69 42 18<br></div></div></div></div></div>
</div>
<br style="font-size: 16px; "></div></body></html>