<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div><span>Zhao, Thank you very much for the answers.</span></div><div><br><span></span></div><div><span>One more question: :)</span></div><div><span><br></span></div><div><span>Do you know how I can calculate this ? Does this method works regardless of the number of nodes allocated (even with any fraction of pset ) ?</span></div><div><br><span></span></div><div><span>Thank you</span></div><div><span>Emalayan</span></div><div><br></div> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> ZHAO ZHANG <zhaozhang@uchicago.edu><br> <b><span style="font-weight: bold;">To:</span></b> Emalayan
Vairavanathan <svemalayan@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> Michael Wilde <wilde@mcs.anl.gov>; Justin M Wozniak <wozniak@mcs.anl.gov>; "swift-devel@ci.uchicago.edu" <swift-devel@ci.uchicago.edu> <br> <b><span style="font-weight: bold;">Sent:</span></b> Sunday, 4 March 2012 4:06 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [Swift-devel] [ZeptoOS] hostname returns none in Surveyor<br> </font> </div> <br><div id="yiv921546743">
<div>
Hi, Emalayan<br>
<br>
On 3/4/2012 6:00 PM, Emalayan Vairavanathan wrote:
<blockquote type="cite">
<div style="color:rgb(0, 0, 0);background-color:rgb(255, 255,
255);font-family:times new roman, new york, times, serif;font-size:12pt;">
<div><span>Mike, that sounds like a good idea.</span></div>
<div><br>
<span></span></div>
<div><span>Zhao</span><span>, In addition to Mike's questions I
have two more questions.</span></div>
<div><br>
<span></span></div>
<div><span>1) Is it possible to get/ calculate the MAX_RANK /
number of nodes in an allocation from persoanlity.h or some
other data structure ?</span></div>
</div>
</blockquote>
Yes, you could calculate the MAX_RANK from personality.sh.<br>
<blockquote type="cite">
<div style="color:rgb(0, 0, 0);background-color:rgb(255, 255,
255);font-family:times new roman, new york, times, serif;font-size:12pt;">
<div><br>
<span></span></div>
<div><span>2) Which interface should be configured to have Tours
? (Does this matter at all ?)</span></div>
<div><span> In your scripts you are configuring eth1. But in
</span> <a rel="nofollow" target="_blank" href="http://wiki.mcs.anl.gov/zeptoos/index.php/Other_Packages">http://wiki.mcs.anl.gov/zeptoos/index.php/Other_Packages</a>
tun1 is configured.</div>
</div>
</blockquote>
To use the torus network, there are two ways. One is to use the
12.x.y.z+1 interface, which we have to configure ourselves.<br>
The other way is to use the "ipfwd.sh", aka the 10.128 interface.
The drawback of the second interface is it takes up one core<br>
for polling, and there is some scalability issue beyond 2K compute
nodes as far as I remember. Mosa could use either of them.<br>
<br>
best<br>
zhao<br>
<blockquote type="cite">
<div style="color:#000;background-color:#fff;font-family:times new roman, new york, times, serif;font-size:12pt;">
<div><br>
</div>
<div>Thank you</div>
<div>Emalayan<br>
</div>
<div><br>
</div>
<div style="font-family:times new roman, new york, times, serif;font-size:12pt;">
<div style="font-family:times new roman, new york, times, serif;font-size:12pt;">
<div dir="ltr"> <font size="2" face="Arial">
<hr size="1"> <b><span style="font-weight:bold;">From:</span></b>
Michael Wilde <a rel="nofollow" class="yiv921546743moz-txt-link-rfc2396E" ymailto="mailto:wilde@mcs.anl.gov" target="_blank" href="mailto:wilde@mcs.anl.gov"><wilde@mcs.anl.gov></a><br>
<b><span style="font-weight:bold;">To:</span></b> ZHAO
ZHANG <a rel="nofollow" class="yiv921546743moz-txt-link-rfc2396E" ymailto="mailto:zhaozhang@uchicago.edu" target="_blank" href="mailto:zhaozhang@uchicago.edu"><zhaozhang@uchicago.edu></a>; Justin M Wozniak
<a rel="nofollow" class="yiv921546743moz-txt-link-rfc2396E" ymailto="mailto:wozniak@mcs.anl.gov" target="_blank" href="mailto:wozniak@mcs.anl.gov"><wozniak@mcs.anl.gov></a> <br>
<b><span style="font-weight:bold;">Cc:</span></b>
Emalayan Vairavanathan <a rel="nofollow" class="yiv921546743moz-txt-link-rfc2396E" ymailto="mailto:svemalayan@yahoo.com" target="_blank" href="mailto:svemalayan@yahoo.com"><svemalayan@yahoo.com></a>;
<a rel="nofollow" class="yiv921546743moz-txt-link-abbreviated" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a> <br>
<b><span style="font-weight:bold;">Sent:</span></b>
Sunday, 4 March 2012 2:33 PM<br>
<b><span style="font-weight:bold;">Subject:</span></b>
Re: [Swift-devel] [ZeptoOS] hostname returns none in
Surveyor<br>
</font> </div>
<br>
Zhao, with this procedure do you get consecutive host IP
addresses starting from 0.0 through 640*64 in the two low
order octets?<br>
<br>
In other words, does your just do what this page describes
under "IP over Torus":<br>
<br>
<a rel="nofollow" target="_blank" href="http://wiki.mcs.anl.gov/zeptoos/index.php/Other_Packages">http://wiki.mcs.anl.gov/zeptoos/index.php/Other_Packages</a><br>
<br>
Is the "ipfwd.sh" script mentioned there still needed, or
does that now happen automatically?<br>
<br>
If so, perhaps we can greatly simplify the Mosa startup: we
need only pass the max rank of the running job, and Mosa
will know that it can use 12.128.0.0 for example. Then we
dont need any manual intervention, nor complicated/brittle
file-waiting logic.<br>
<br>
Zhao, I dont understand why your example is using the
12.0.0.0 network vs the example on the page above which uses
10.128.0.0. Can you help me understand what is going on
here? Is the "IP Over Torus" info on the ZeptoOS wiki
outdated? Or does it describe a different technique?<br>
<br>
Justin, have you also mastered similar techniques for JETS?
Do we need help form the ZeptoOS team on this?<br>
<br>
Thanks,<br>
<br>
- Mike<br>
<br>
<br>
<br>
----- Original Message -----<br>
> From: "ZHAO ZHANG" <<a rel="nofollow" ymailto="mailto:zhaozhang@uchicago.edu" target="_blank" href="mailto:zhaozhang@uchicago.edu">zhaozhang@uchicago.edu</a>><br>
> To: "Michael Wilde" <<a rel="nofollow" ymailto="mailto:wilde@mcs.anl.gov" target="_blank" href="mailto:wilde@mcs.anl.gov">wilde@mcs.anl.gov</a>><br>
> Cc: "Emalayan Vairavanathan" <<a rel="nofollow" ymailto="mailto:svemalayan@yahoo.com" target="_blank" href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>>,
<a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a><br>
> Sent: Sunday, March 4, 2012 1:17:18 PM<br>
> Subject: Re: [Swift-devel] [ZeptoOS] hostname returns
none in Surveyor<br>
> Yes, each compute node needs to run this script to
bring up the<br>
> network<br>
> interface.<br>
> <br>
> zhao<br>
> <br>
> On 3/4/2012 12:53 PM, Michael Wilde wrote:<br>
> > Thanks, Zhao. Does this need to run on each node
at startup?<br>
> ><br>
> > If so should this logic be integrated into the
worker startup<br>
> > script, Jon, Justin, Emalayan?<br>
> ><br>
> > Ive not looked at the current scripts much; I
would think that all<br>
> > the BG/P specific logic of enabling the torus
network and finding<br>
> > each node's IP address on the torus should be done
in the init<br>
> > script rather than in the worker.<br>
> ><br>
> > - Mike<br>
> ><br>
> > ----- Original Message -----<br>
> >> From: "ZHAO ZHANG"<<a rel="nofollow" ymailto="mailto:zhaozhang@uchicago.edu" target="_blank" href="mailto:zhaozhang@uchicago.edu">zhaozhang@uchicago.edu</a>><br>
> >> To: "Michael Wilde"<<a rel="nofollow" ymailto="mailto:wilde@mcs.anl.gov" target="_blank" href="mailto:wilde@mcs.anl.gov">wilde@mcs.anl.gov</a>><br>
> >> Cc: "Emalayan Vairavanathan"<<a rel="nofollow" ymailto="mailto:svemalayan@yahoo.com" target="_blank" href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>>,<br>
> >> <a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a><br>
> >> Sent: Sunday, March 4, 2012 12:18:28 PM<br>
> >> Subject: Re: [Swift-devel] [ZeptoOS] hostname
returns none in<br>
> >> Surveyor<br>
> >> Hi, Mike<br>
> >><br>
> >> With 192.168.1.*, we could only access the
tree network. In order<br>
> >> to<br>
> >> use<br>
> >> the torus network, we need to use the
12.x.y.z+1 ip address. (x, y,<br>
> >> z<br>
> >> here is the coordinates of the compute nodes).<br>
> >> The code below could bring the torus ip
address up.<br>
> >><br>
> >> IP=""<br>
> >> set_torus_ip()<br>
> >> {<br>
> >> x=$1<br>
> >> y=$2<br>
> >> z=$3<br>
> >> z=`expr $3 + 1`<br>
> >> ifconfig eth1 12.$x.$y.$z netmask 255.0.0.0
mtu 8996 -arp<br>
> >> IP=12.$x.$y.$z<br>
> >> }<br>
> >> BG_PSETORG=`cat /proc/personality.sh | grep
BG_PSETORG | cut -d '"'<br>
> >> -f<br>
> >> 2`<br>
> >> echo ${BG_PSETORG}>> /dev/shm/localip<br>
> >> set_torus_ip $BG_PSETORG<br>
> >><br>
> >> best<br>
> >> zhao<br>
> >><br>
> >> On 3/4/2012 10:24 AM, Michael Wilde wrote:<br>
> >>> Zhao,<br>
> >>><br>
> >>> Can you tell us if the nodes on the torus
network are accessed<br>
> >>> over<br>
> >>> the 192.168 network? I just realized they
cant all be on the<br>
> >>> 192.168.1 subnet, so I hope I suggested
the right network here.<br>
> >>><br>
> >>> Thanks,<br>
> >>><br>
> >>> - Mike<br>
> >>><br>
> >>> ----- Original Message -----<br>
> >>>> From: "Emalayan Vairavanathan"<<a rel="nofollow" ymailto="mailto:svemalayan@yahoo.com" target="_blank" href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>><br>
> >>>> To: <a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a><br>
> >>>> Sent: Sunday, March 4, 2012 1:40:53 AM<br>
> >>>> Subject: Re: [Swift-devel] [ZeptoOS]
hostname returns none in<br>
> >>>> Surveyor<br>
> >>>> Thank you very much Mike. I agree with
you suggestion. I can do<br>
> >>>> that<br>
> >>>> in worker.pl.<br>
> >>>><br>
> >>>><br>
> >>>> Thank you<br>
> >>>> Emalayan<br>
> >>>><br>
> >>>><br>
> >>>><br>
> >>>><br>
> >>>><br>
> >>>><br>
> >>>> From: Michael Wilde<<a rel="nofollow" ymailto="mailto:wilde@mcs.anl.gov" target="_blank" href="mailto:wilde@mcs.anl.gov">wilde@mcs.anl.gov</a>><br>
> >>>> To: <a rel="nofollow" ymailto="mailto:emalayan@ece.ubc.ca" target="_blank" href="mailto:emalayan@ece.ubc.ca">emalayan@ece.ubc.ca</a><br>
> >>>> Cc: swift-devel<<a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a>><br>
> >>>> Sent: Saturday, 3 March 2012 7:39 PM<br>
> >>>> Subject: Re: [Swift-devel] [ZeptoOS]
hostname returns none in<br>
> >>>> Surveyor<br>
> >>>><br>
> >>>> Emalayan,<br>
> >>>><br>
> >>>> I wasnt paying much attention to the
actual IP address returned<br>
> >>>> by<br>
> >>>> hostname in the zeptoos profile.<br>
> >>>><br>
> >>>> Since these are the addresses that
Mosa will communicate over, I<br>
> >>>> think<br>
> >>>> you *do* want them to be the
192.168.1.* addresses of the nodes<br>
> >>>> on<br>
> >>>> the<br>
> >>>> torus network (in other words tun0).<br>
> >>>><br>
> >>>> So, since both profiles return
192.168.1.64 for the tun0 IP, I<br>
> >>>> think<br>
> >>>> thats what you should use. So try
replacing `hostname` in<br>
> >>>> worker.pl<br>
> >>>> with something like:<br>
> >>>><br>
> >>>> `ifconfig | grep 192.168 | sed -e
's/^inet addr://' -e 's/ .*//'`<br>
> >>>><br>
> >>>> You may have to adapt this a bit to
meet your needs. Im assuming<br>
> >>>> that<br>
> >>>> the only code that will uses these IPs
is MosaStore.<br>
> >>>><br>
> >>>> - Mike<br>
> >>>><br>
> >>>><br>
> >>>> ----- Original Message -----<br>
> >>>>> From: "Kazutomo Yoshii"< <a rel="nofollow" ymailto="mailto:kazutomo@mcs.anl.gov" target="_blank" href="mailto:kazutomo@mcs.anl.gov">kazutomo@mcs.anl.gov</a>><br>
> >>>>> To: <a rel="nofollow" ymailto="mailto:zeptoos@lists.mcs.anl.gov" target="_blank" href="mailto:zeptoos@lists.mcs.anl.gov">zeptoos@lists.mcs.anl.gov</a><br>
> >>>>> Sent: Saturday, March 3, 2012
8:52:00 PM<br>
> >>>>> Subject: Re: [ZeptoOS] hostname
returns none in Surveyor<br>
> >>>>> Hi Emalayan,<br>
> >>>>><br>
> >>>>> The zeptoos profile returns the IP
address of associated I/O<br>
> >>>>> node,<br>
> >>>>> which is kind of wrong in my
opinion (influence of IBM CNK).<br>
> >>>>> ifconfig on compute nodes returns
CN's IP address, which is<br>
> >>>>> correct.<br>
> >>>>> e.g. tun0 192.168.1.64<br>
> >>>>><br>
> >>>>> If you want to find associated
ION's IP address from CNs,<br>
> >>>>> do something like this.<br>
> >>>>><br>
> >>>>> $ grep BG_IP= /proc/personality.sh<br>
> >>>>><br>
> >>>>> - kaz<br>
> >>>>><br>
> >>>>> On 03/03/2012 08:25 PM, Emalayan
Vairavanathan wrote:<br>
> >>>>>> Hi All,<br>
> >>>>>><br>
> >>>>>> I am trying to run some
experiments in Surveyor. The software I<br>
> >>>>>> am<br>
> >>>>>> using<br>
> >>>>>> gets the IP-address of
compute-nodes using hostname command.<br>
> >>>>>><br>
> >>>>>> With zepto-vn-eval/mosatest
profile hostname command returns<br>
> >>>>>> none.<br>
> >>>>>> But with zeptoos profile
hostname returns the correct IP<br>
> >>>>>> address.<br>
> >>>>>><br>
> >>>>>> Is this due to some
configuration issues in<br>
> >>>>>> zepto-vn-eval/mosatest<br>
> >>>>>> profile?As a workaround I
tired to use ifconfig with both<br>
> >>>>>> profiles,<br>
> >>>>>> but<br>
> >>>>>> it seems ifconfig is not
returning the correct IP address.<br>
> >>>>>><br>
> >>>>>> Is there any command / files
which I can used to retrieve the<br>
> >>>>>> hostname<br>
> >>>>>> on compute nodes? I have
pasted the console output with both<br>
> >>>>>> profiles<br>
> >>>>>> below. Please let me know if
you need more details.<br>
> >>>>>><br>
> >>>>>> Thank you<br>
> >>>>>> Emalayan<br>
> >>>>>><br>
> >>>>>><br>
> >>>>>> =======================With
zeptoos profile<br>
> >>>>>>
===============================<br>
> >>>>>><br>
> >>>>>> / # hostname<br>
> >>>>>> 172.18.3.19<br>
> >>>>>> / #<br>
> >>>>>> / # cat
/proc/sys/kernel/hostname<br>
> >>>>>> 172.18.3.19<br>
> >>>>>> / #<br>
> >>>>>> / #<br>
> >>>>>> / # ifconfig -a<br>
> >>>>>> lo Link encap:Local Loopback<br>
> >>>>>> inet addr:127.0.0.1
Mask:255.0.0.0<br>
> >>>>>> UP LOOPBACK RUNNING MTU:16436
Metric:1<br>
> >>>>>> RX packets:0 errors:0
dropped:0 overruns:0 frame:0<br>
> >>>>>> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0<br>
> >>>>>> collisions:0 txqueuelen:0<br>
> >>>>>> RX bytes:0 (0.0 B) TX bytes:0
(0.0 B)<br>
> >>>>>><br>
> >>>>>> tun0 Link encap:UNSPEC HWaddr<br>
> >>>>>>
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00<br>
> >>>>>> inet addr:192.168.1.64
P-t-P:192.168.1.254 Mask:255.255.255.255<br>
> >>>>>> UP POINTOPOINT RUNNING NOARP
MULTICAST MTU:65535 Metric:1<br>
> >>>>>> RX packets:2662 errors:0
dropped:0 overruns:0 frame:0<br>
> >>>>>> TX packets:1772 errors:0
dropped:0 overruns:0 carrier:0<br>
> >>>>>> collisions:0 txqueuelen:500<br>
> >>>>>> RX bytes:140206 (136.9 KiB) TX
bytes:125412 (122.4 KiB)<br>
> >>>>>><br>
> >>>>>><br>
> >>>>>><br>
> >>>>>> =======================With
zepto-vn-eval/mosatest profile<br>
> >>>>>>
===============================<br>
> >>>>>><br>
> >>>>>> /etc # hostname<br>
> >>>>>> (none)<br>
> >>>>>> /etc #<br>
> >>>>>> /etc # cat
/proc/sys/kernel/hostname<br>
> >>>>>> (none)<br>
> >>>>>> /etc #<br>
> >>>>>> /etc # ifconfig -a<br>
> >>>>>> eth0 Link encap:Ethernet
HWaddr 00:80:46:00:00:00<br>
> >>>>>> BROADCAST MULTICAST MTU:1500
Metric:1<br>
> >>>>>> RX packets:0 errors:0
dropped:0 overruns:0 frame:0<br>
> >>>>>> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0<br>
> >>>>>> collisions:0 txqueuelen:1000<br>
> >>>>>> RX bytes:0 (0.0 B) TX bytes:0
(0.0 B)<br>
> >>>>>><br>
> >>>>>> eth1 Link encap:Ethernet
HWaddr 00:80:47:00:00:00<br>
> >>>>>> BROADCAST MULTICAST MTU:1500
Metric:1<br>
> >>>>>> RX packets:0 errors:0
dropped:0 overruns:0 frame:0<br>
> >>>>>> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0<br>
> >>>>>> collisions:0 txqueuelen:1000<br>
> >>>>>> RX bytes:0 (0.0 B) TX bytes:0
(0.0 B)<br>
> >>>>>><br>
> >>>>>> lo Link encap:Local Loopback<br>
> >>>>>> inet addr:127.0.0.1
Mask:255.0.0.0<br>
> >>>>>> inet6 addr: ::1/128 Scope:Host<br>
> >>>>>> UP LOOPBACK RUNNING MTU:16436
Metric:1<br>
> >>>>>> RX packets:0 errors:0
dropped:0 overruns:0 frame:0<br>
> >>>>>> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0<br>
> >>>>>> collisions:0 txqueuelen:0<br>
> >>>>>> RX bytes:0 (0.0 B) TX bytes:0
(0.0 B)<br>
> >>>>>><br>
> >>>>>> sit0 Link encap:IPv6-in-IPv4<br>
> >>>>>> NOARP MTU:1480 Metric:1<br>
> >>>>>> RX packets:0 errors:0
dropped:0 overruns:0 frame:0<br>
> >>>>>> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0<br>
> >>>>>> collisions:0 txqueuelen:0<br>
> >>>>>> RX bytes:0 (0.0 B) TX bytes:0
(0.0 B)<br>
> >>>>>><br>
> >>>>>> tun0 Link encap:UNSPEC HWaddr<br>
> >>>>>>
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00<br>
> >>>>>> inet addr:192.168.1.64
P-t-P:192.168.1.254 Mask:255.255.255.255<br>
> >>>>>> UP POINTOPOINT RUNNING NOARP
MULTICAST MTU:65535 Metric:1<br>
> >>>>>> RX packets:965 errors:0
dropped:0 overruns:0 frame:0<br>
> >>>>>> TX packets:627 errors:0
dropped:0 overruns:0 carrier:0<br>
> >>>>>> collisions:0 txqueuelen:500<br>
> >>>>>> RX bytes:50984 (49.7 KiB) TX
bytes:50530 (49.3 KiB)<br>
> >>>>>><br>
> >>>> --<br>
> >>>> Michael Wilde<br>
> >>>> Computation Institute, University of
Chicago<br>
> >>>> Mathematics and Computer Science
Division<br>
> >>>> Argonne National Laboratory<br>
> >>>><br>
> >>>>
_______________________________________________<br>
> >>>> Swift-devel mailing list<br>
> >>>> <a rel="nofollow" ymailto="mailto:Swift-devel@ci.uchicago.edu" target="_blank" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>
> >>>> <a rel="nofollow" target="_blank" href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>
> >>>><br>
> >>>><br>
> >>>><br>
> >>>>
_______________________________________________<br>
> >>>> Swift-devel mailing list<br>
> >>>> <a rel="nofollow" ymailto="mailto:Swift-devel@ci.uchicago.edu" target="_blank" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>
> >>>> <a rel="nofollow" target="_blank" href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>
<br>
-- <br>
Michael Wilde<br>
Computation Institute, University of Chicago<br>
Mathematics and Computer Science Division<br>
Argonne National Laboratory<br>
<br>
<br>
<br>
</div>
</div>
</div>
</blockquote>
</div>
</div><br><br> </div> </div> </div></body></html>