<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div><span>Mike, that sounds like a good idea.</span></div><div><br><span></span></div><div><span>Zhao</span><span>, In addition to Mike's questions I have two more questions.</span></div><div><br><span></span></div><div><span>1) Is it possible to get/ calculate the MAX_RANK / number of nodes in an allocation from persoanlity.h or some other data structure ?</span></div><div><br><span></span></div><div><span>2) Which interface should be configured to have Tours ? (Does this matter at all ?)</span></div><div><span> In your scripts you are configuring eth1. But in </span> <a href="http://wiki.mcs.anl.gov/zeptoos/index.php/Other_Packages" target="_blank">http://wiki.mcs.anl.gov/zeptoos/index.php/Other_Packages</a> tun1 is configured.</div><div><br></div><div>Thank
you</div><div>Emalayan<br></div><div><br></div> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Michael Wilde <wilde@mcs.anl.gov><br> <b><span style="font-weight: bold;">To:</span></b> ZHAO ZHANG <zhaozhang@uchicago.edu>; Justin M Wozniak <wozniak@mcs.anl.gov> <br><b><span style="font-weight: bold;">Cc:</span></b> Emalayan Vairavanathan <svemalayan@yahoo.com>; swift-devel@ci.uchicago.edu <br> <b><span style="font-weight: bold;">Sent:</span></b> Sunday, 4 March 2012 2:33 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [Swift-devel] [ZeptoOS] hostname returns none in Surveyor<br> </font> </div> <br>Zhao, with this procedure do you get consecutive host IP addresses starting from 0.0
through 640*64 in the two low order octets?<br><br>In other words, does your just do what this page describes under "IP over Torus":<br><br> <a href="http://wiki.mcs.anl.gov/zeptoos/index.php/Other_Packages" target="_blank">http://wiki.mcs.anl.gov/zeptoos/index.php/Other_Packages</a><br><br>Is the "ipfwd.sh" script mentioned there still needed, or does that now happen automatically?<br><br>If so, perhaps we can greatly simplify the Mosa startup: we need only pass the max rank of the running job, and Mosa will know that it can use 12.128.0.0 for example. Then we dont need any manual intervention, nor complicated/brittle file-waiting logic.<br><br>Zhao, I dont understand why your example is using the 12.0.0.0 network vs the example on the page above which uses 10.128.0.0. Can you help me understand what is going on here? Is the "IP Over Torus" info on the ZeptoOS wiki outdated? Or does it describe a different technique?<br><br>Justin,
have you also mastered similar techniques for JETS? Do we need help form the ZeptoOS team on this?<br><br>Thanks,<br><br>- Mike<br><br> <br><br>----- Original Message -----<br>> From: "ZHAO ZHANG" <<a ymailto="mailto:zhaozhang@uchicago.edu" href="mailto:zhaozhang@uchicago.edu">zhaozhang@uchicago.edu</a>><br>> To: "Michael Wilde" <<a ymailto="mailto:wilde@mcs.anl.gov" href="mailto:wilde@mcs.anl.gov">wilde@mcs.anl.gov</a>><br>> Cc: "Emalayan Vairavanathan" <<a ymailto="mailto:svemalayan@yahoo.com" href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>>, <a ymailto="mailto:swift-devel@ci.uchicago.edu" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a><br>> Sent: Sunday, March 4, 2012 1:17:18 PM<br>> Subject: Re: [Swift-devel] [ZeptoOS] hostname returns none in Surveyor<br>> Yes, each compute node needs to run this script to bring up the<br>> network<br>> interface.<br>> <br>>
zhao<br>> <br>> On 3/4/2012 12:53 PM, Michael Wilde wrote:<br>> > Thanks, Zhao. Does this need to run on each node at startup?<br>> ><br>> > If so should this logic be integrated into the worker startup<br>> > script, Jon, Justin, Emalayan?<br>> ><br>> > Ive not looked at the current scripts much; I would think that all<br>> > the BG/P specific logic of enabling the torus network and finding<br>> > each node's IP address on the torus should be done in the init<br>> > script rather than in the worker.<br>> ><br>> > - Mike<br>> ><br>> > ----- Original Message -----<br>> >> From: "ZHAO ZHANG"<<a ymailto="mailto:zhaozhang@uchicago.edu" href="mailto:zhaozhang@uchicago.edu">zhaozhang@uchicago.edu</a>><br>> >> To: "Michael Wilde"<<a ymailto="mailto:wilde@mcs.anl.gov" href="mailto:wilde@mcs.anl.gov">wilde@mcs.anl.gov</a>><br>> >> Cc:
"Emalayan Vairavanathan"<<a ymailto="mailto:svemalayan@yahoo.com" href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>>,<br>> >> <a ymailto="mailto:swift-devel@ci.uchicago.edu" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a><br>> >> Sent: Sunday, March 4, 2012 12:18:28 PM<br>> >> Subject: Re: [Swift-devel] [ZeptoOS] hostname returns none in<br>> >> Surveyor<br>> >> Hi, Mike<br>> >><br>> >> With 192.168.1.*, we could only access the tree network. In order<br>> >> to<br>> >> use<br>> >> the torus network, we need to use the 12.x.y.z+1 ip address. (x, y,<br>> >> z<br>> >> here is the coordinates of the compute nodes).<br>> >> The code below could bring the torus ip address up.<br>> >><br>> >> IP=""<br>> >> set_torus_ip()<br>> >> {<br>> >> x=$1<br>>
>> y=$2<br>> >> z=$3<br>> >> z=`expr $3 + 1`<br>> >> ifconfig eth1 12.$x.$y.$z netmask 255.0.0.0 mtu 8996 -arp<br>> >> IP=12.$x.$y.$z<br>> >> }<br>> >> BG_PSETORG=`cat /proc/personality.sh | grep BG_PSETORG | cut -d '"'<br>> >> -f<br>> >> 2`<br>> >> echo ${BG_PSETORG}>> /dev/shm/localip<br>> >> set_torus_ip $BG_PSETORG<br>> >><br>> >> best<br>> >> zhao<br>> >><br>> >> On 3/4/2012 10:24 AM, Michael Wilde wrote:<br>> >>> Zhao,<br>> >>><br>> >>> Can you tell us if the nodes on the torus network are accessed<br>> >>> over<br>> >>> the 192.168 network? I just realized they cant all be on the<br>> >>> 192.168.1 subnet, so I hope I suggested the right network here.<br>> >>><br>> >>> Thanks,<br>> >>><br>>
>>> - Mike<br>> >>><br>> >>> ----- Original Message -----<br>> >>>> From: "Emalayan Vairavanathan"<<a ymailto="mailto:svemalayan@yahoo.com" href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>><br>> >>>> To: <a ymailto="mailto:swift-devel@ci.uchicago.edu" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a><br>> >>>> Sent: Sunday, March 4, 2012 1:40:53 AM<br>> >>>> Subject: Re: [Swift-devel] [ZeptoOS] hostname returns none in<br>> >>>> Surveyor<br>> >>>> Thank you very much Mike. I agree with you suggestion. I can do<br>> >>>> that<br>> >>>> in worker.pl.<br>> >>>><br>> >>>><br>> >>>> Thank you<br>> >>>> Emalayan<br>> >>>><br>> >>>><br>> >>>><br>>
>>>><br>> >>>><br>> >>>><br>> >>>> From: Michael Wilde<<a ymailto="mailto:wilde@mcs.anl.gov" href="mailto:wilde@mcs.anl.gov">wilde@mcs.anl.gov</a>><br>> >>>> To: <a ymailto="mailto:emalayan@ece.ubc.ca" href="mailto:emalayan@ece.ubc.ca">emalayan@ece.ubc.ca</a><br>> >>>> Cc: swift-devel<<a ymailto="mailto:swift-devel@ci.uchicago.edu" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a>><br>> >>>> Sent: Saturday, 3 March 2012 7:39 PM<br>> >>>> Subject: Re: [Swift-devel] [ZeptoOS] hostname returns none in<br>> >>>> Surveyor<br>> >>>><br>> >>>> Emalayan,<br>> >>>><br>> >>>> I wasnt paying much attention to the actual IP address returned<br>> >>>> by<br>> >>>> hostname in the zeptoos profile.<br>>
>>>><br>> >>>> Since these are the addresses that Mosa will communicate over, I<br>> >>>> think<br>> >>>> you *do* want them to be the 192.168.1.* addresses of the nodes<br>> >>>> on<br>> >>>> the<br>> >>>> torus network (in other words tun0).<br>> >>>><br>> >>>> So, since both profiles return 192.168.1.64 for the tun0 IP, I<br>> >>>> think<br>> >>>> thats what you should use. So try replacing `hostname` in<br>> >>>> worker.pl<br>> >>>> with something like:<br>> >>>><br>> >>>> `ifconfig | grep 192.168 | sed -e 's/^inet addr://' -e 's/ .*//'`<br>> >>>><br>> >>>> You may have to adapt this a bit to meet your needs. Im assuming<br>> >>>> that<br>> >>>> the only code that will
uses these IPs is MosaStore.<br>> >>>><br>> >>>> - Mike<br>> >>>><br>> >>>><br>> >>>> ----- Original Message -----<br>> >>>>> From: "Kazutomo Yoshii"< <a ymailto="mailto:kazutomo@mcs.anl.gov" href="mailto:kazutomo@mcs.anl.gov">kazutomo@mcs.anl.gov</a>><br>> >>>>> To: <a ymailto="mailto:zeptoos@lists.mcs.anl.gov" href="mailto:zeptoos@lists.mcs.anl.gov">zeptoos@lists.mcs.anl.gov</a><br>> >>>>> Sent: Saturday, March 3, 2012 8:52:00 PM<br>> >>>>> Subject: Re: [ZeptoOS] hostname returns none in Surveyor<br>> >>>>> Hi Emalayan,<br>> >>>>><br>> >>>>> The zeptoos profile returns the IP address of associated I/O<br>> >>>>> node,<br>> >>>>> which is kind of wrong in my opinion (influence of IBM CNK).<br>>
>>>>> ifconfig on compute nodes returns CN's IP address, which is<br>> >>>>> correct.<br>> >>>>> e.g. tun0 192.168.1.64<br>> >>>>><br>> >>>>> If you want to find associated ION's IP address from CNs,<br>> >>>>> do something like this.<br>> >>>>><br>> >>>>> $ grep BG_IP= /proc/personality.sh<br>> >>>>><br>> >>>>> - kaz<br>> >>>>><br>> >>>>> On 03/03/2012 08:25 PM, Emalayan Vairavanathan wrote:<br>> >>>>>> Hi All,<br>> >>>>>><br>> >>>>>> I am trying to run some experiments in Surveyor. The software I<br>> >>>>>> am<br>> >>>>>> using<br>> >>>>>> gets the IP-address of compute-nodes using hostname command.<br>>
>>>>>><br>> >>>>>> With zepto-vn-eval/mosatest profile hostname command returns<br>> >>>>>> none.<br>> >>>>>> But with zeptoos profile hostname returns the correct IP<br>> >>>>>> address.<br>> >>>>>><br>> >>>>>> Is this due to some configuration issues in<br>> >>>>>> zepto-vn-eval/mosatest<br>> >>>>>> profile?As a workaround I tired to use ifconfig with both<br>> >>>>>> profiles,<br>> >>>>>> but<br>> >>>>>> it seems ifconfig is not returning the correct IP address.<br>> >>>>>><br>> >>>>>> Is there any command / files which I can used to retrieve the<br>> >>>>>> hostname<br>> >>>>>> on compute nodes? I have pasted the console
output with both<br>> >>>>>> profiles<br>> >>>>>> below. Please let me know if you need more details.<br>> >>>>>><br>> >>>>>> Thank you<br>> >>>>>> Emalayan<br>> >>>>>><br>> >>>>>><br>> >>>>>> =======================With zeptoos profile<br>> >>>>>> ===============================<br>> >>>>>><br>> >>>>>> / # hostname<br>> >>>>>> 172.18.3.19<br>> >>>>>> / #<br>> >>>>>> / # cat /proc/sys/kernel/hostname<br>> >>>>>> 172.18.3.19<br>> >>>>>> / #<br>> >>>>>> / #<br>> >>>>>> / # ifconfig -a<br>> >>>>>> lo Link encap:Local Loopback<br>> >>>>>> inet
addr:127.0.0.1 Mask:255.0.0.0<br>> >>>>>> UP LOOPBACK RUNNING MTU:16436 Metric:1<br>> >>>>>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br>> >>>>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br>> >>>>>> collisions:0 txqueuelen:0<br>> >>>>>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)<br>> >>>>>><br>> >>>>>> tun0 Link encap:UNSPEC HWaddr<br>> >>>>>> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00<br>> >>>>>> inet addr:192.168.1.64 P-t-P:192.168.1.254 Mask:255.255.255.255<br>> >>>>>> UP POINTOPOINT RUNNING NOARP MULTICAST MTU:65535 Metric:1<br>> >>>>>> RX packets:2662 errors:0 dropped:0 overruns:0 frame:0<br>> >>>>>> TX packets:1772 errors:0 dropped:0 overruns:0 carrier:0<br>>
>>>>>> collisions:0 txqueuelen:500<br>> >>>>>> RX bytes:140206 (136.9 KiB) TX bytes:125412 (122.4 KiB)<br>> >>>>>><br>> >>>>>><br>> >>>>>><br>> >>>>>> =======================With zepto-vn-eval/mosatest profile<br>> >>>>>> ===============================<br>> >>>>>><br>> >>>>>> /etc # hostname<br>> >>>>>> (none)<br>> >>>>>> /etc #<br>> >>>>>> /etc # cat /proc/sys/kernel/hostname<br>> >>>>>> (none)<br>> >>>>>> /etc #<br>> >>>>>> /etc # ifconfig -a<br>> >>>>>> eth0 Link encap:Ethernet HWaddr 00:80:46:00:00:00<br>> >>>>>> BROADCAST MULTICAST MTU:1500 Metric:1<br>> >>>>>> RX packets:0
errors:0 dropped:0 overruns:0 frame:0<br>> >>>>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br>> >>>>>> collisions:0 txqueuelen:1000<br>> >>>>>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)<br>> >>>>>><br>> >>>>>> eth1 Link encap:Ethernet HWaddr 00:80:47:00:00:00<br>> >>>>>> BROADCAST MULTICAST MTU:1500 Metric:1<br>> >>>>>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br>> >>>>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br>> >>>>>> collisions:0 txqueuelen:1000<br>> >>>>>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)<br>> >>>>>><br>> >>>>>> lo Link encap:Local Loopback<br>> >>>>>> inet addr:127.0.0.1 Mask:255.0.0.0<br>> >>>>>> inet6 addr: ::1/128
Scope:Host<br>> >>>>>> UP LOOPBACK RUNNING MTU:16436 Metric:1<br>> >>>>>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br>> >>>>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br>> >>>>>> collisions:0 txqueuelen:0<br>> >>>>>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)<br>> >>>>>><br>> >>>>>> sit0 Link encap:IPv6-in-IPv4<br>> >>>>>> NOARP MTU:1480 Metric:1<br>> >>>>>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br>> >>>>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br>> >>>>>> collisions:0 txqueuelen:0<br>> >>>>>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)<br>> >>>>>><br>> >>>>>> tun0 Link encap:UNSPEC HWaddr<br>> >>>>>>
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00<br>> >>>>>> inet addr:192.168.1.64 P-t-P:192.168.1.254 Mask:255.255.255.255<br>> >>>>>> UP POINTOPOINT RUNNING NOARP MULTICAST MTU:65535 Metric:1<br>> >>>>>> RX packets:965 errors:0 dropped:0 overruns:0 frame:0<br>> >>>>>> TX packets:627 errors:0 dropped:0 overruns:0 carrier:0<br>> >>>>>> collisions:0 txqueuelen:500<br>> >>>>>> RX bytes:50984 (49.7 KiB) TX bytes:50530 (49.3 KiB)<br>> >>>>>><br>> >>>> --<br>> >>>> Michael Wilde<br>> >>>> Computation Institute, University of Chicago<br>> >>>> Mathematics and Computer Science Division<br>> >>>> Argonne National Laboratory<br>> >>>><br>> >>>> _______________________________________________<br>>
>>>> Swift-devel mailing list<br>> >>>> <a ymailto="mailto:Swift-devel@ci.uchicago.edu" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>> >>>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>> >>>><br>> >>>><br>> >>>><br>> >>>> _______________________________________________<br>> >>>> Swift-devel mailing list<br>> >>>> <a ymailto="mailto:Swift-devel@ci.uchicago.edu" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>> >>>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br><br>-- <br>Michael Wilde<br>Computation Institute,
University of Chicago<br>Mathematics and Computer Science Division<br>Argonne National Laboratory<br><br><br><br> </div> </div> </div></body></html>