<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div>Hi Matei,</div><div><br></div><div>I am currently working on evaluating the gains of locality on BG/P. I will be able to get some numbers today / tomorrow. This will help us in taking decisions.</div><div><br></div><div>Thank you</div><div>Emalayan.<br></div><div><span class="yui_3_2_0_147_1332638993418177" style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"><br></span></div><div><br></div> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div dir="ltr"> <font face="Arial" size="2"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Matei Ripeanu <matei.ripeanu@gmail.com><br> <b><span style="font-weight: bold;">To:</span></b> 'Emalayan Vairavanathan'
<svemalayan@yahoo.com>; mosastore@googlegroups.com; 'matei' <matei@ece.ubc.ca> <br><b><span style="font-weight: bold;">Cc:</span></b> swift-devel@ci.uchicago.edu <br> <b><span style="font-weight: bold;">Sent:</span></b> Friday, 30 March 2012 4:04 AM<br> <b><span style="font-weight: bold;">Subject:</span></b> RE: [Swift-devel] Data-aware scheduling in Swift ?<br> </font> </div> <br><div id="yiv2016430077"><style><!--
#yiv2016430077
_filtered #yiv2016430077 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}
_filtered #yiv2016430077 {font-family:Tahoma;panose-1:2 11 6 4 3 5 4 4 2 4;}
#yiv2016430077
#yiv2016430077 p.yiv2016430077MsoNormal, #yiv2016430077 li.yiv2016430077MsoNormal, #yiv2016430077 div.yiv2016430077MsoNormal
{margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;font-family:"serif";}
#yiv2016430077 a:link, #yiv2016430077 span.yiv2016430077MsoHyperlink
{color:blue;text-decoration:underline;}
#yiv2016430077 a:visited, #yiv2016430077 span.yiv2016430077MsoHyperlinkFollowed
{color:purple;text-decoration:underline;}
#yiv2016430077 p.yiv2016430077MsoAcetate, #yiv2016430077 li.yiv2016430077MsoAcetate, #yiv2016430077 div.yiv2016430077MsoAcetate
{margin:0cm;margin-bottom:.0001pt;font-size:8.0pt;font-family:"sans-serif";}
#yiv2016430077 span.yiv2016430077EmailStyle17
{font-family:"sans-serif";color:#1F497D;}
#yiv2016430077 span.yiv2016430077BalloonTextChar
{font-family:"sans-serif";}
#yiv2016430077 .yiv2016430077MsoChpDefault
{font-size:10.0pt;}
_filtered #yiv2016430077 {margin:72.0pt 72.0pt 72.0pt 72.0pt;}
#yiv2016430077 div.yiv2016430077WordSection1
{}
--></style><div><div class="yiv2016430077WordSection1"><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;">Emalayan, Mike, Justin, all,</span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;">There are a number of points worth discussing before we fully embark into this: </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;">First: We need to better understand what the gains we expect to have on BG/P from locality. We know we have sizeable gains on our cluster with
data stored on disk (and where we have much lower cross-section bandwidth). I expect that most of these gains are preserved when we use RAM disks on our cluster. And will stay there as long as we do not have to transfer huge volumes of data. Unfortunately we can test this only with 20 nodes - I have no good intuition about what will happen o BG/P at large scale.</span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;">Second: We should discuss how key is having this feature on Swift on BG/P for all the other points we want to prove for the paper. I think support for only one of the patterns we look at to optimize with the cross-layer communication can be demonstrated without (e.g., the one for broadcast) while the other two
(pipelines and gather) can not. On the other side, is there a way to run our benchmark scripts on BG/P (I guess not) to demonstrate the potential gains if Swift implemented that? Or can we run (some of) the applications without Swift on our cluster?</span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;">Third: I am afraid getting functionality this into Swift/Coasters is quite some work. On the other side Mike suggests a relatively clear implementation path. (It will probably work for pipelines but I’m not sure it will work for ‘gather’)</span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div
class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;">What I suggest: Let’s discuss between ourselves three things before embarking into changing Swift/Coasters: (1) we want to increase the certainty that we’ll see performance gains if we implement this, (2) see whether there aren’t ways to demonstrate (some of) what we want outside Swift; (3) re-evaluate the schedule and priorities – we have roughly four weeks to the deadline.</span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;">Let me know what you think,</span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;">
</span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;">-Matei </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span
style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></div><div class="yiv2016430077MsoNormal"><a rel="nofollow" name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"sans-serif";color:#1F497D;"> </span></a></div><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm;"><div class="yiv2016430077MsoNormal"><b><span style="font-size:10.0pt;font-family:"sans-serif";" lang="EN-US">From:</span></b><span style="font-size:10.0pt;font-family:"sans-serif";" lang="EN-US"> Emalayan Vairavanathan [mailto:svemalayan@yahoo.com] <br><b>Sent:</b> March-29-12 7:06 PM<br><b>To:</b>
mosastore@googlegroups.com; matei<br><b>Cc:</b> swift-devel@ci.uchicago.edu<br><b>Subject:</b> Re: [Swift-devel] Data-aware scheduling in Swift ?</span></div></div></div><div class="yiv2016430077MsoNormal"> </div><div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:black;">Thank you Jon, Mike and Justin.</span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:black;"> </span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:black;">Having this functionality would be really useful for us to demonstrate how useful extended attributes are in MosaStore in long term. Further for our SC paper this is a critical functionality and we need this to support both pipeline and reduce patters.</span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:black;">
</span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:#0000BF;">Mike:</span><span style="color:black;"> I will be happy to help with this. In terms of effort and priories, how much time we need to spend to get this done? Is it feasible to target this for our SC paper ? </span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:black;"> </span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:#0000BF;">Justin: </span><span style="color:black;">We do have numbers for the difference between a local MosaStore access and a remote access on our cluster. This is what we have published in CCGrid 2012 (I have attached the paper). But we do not have numbers on BG/P. I can try it on BG/P and get back to you.</span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span
style="color:black;"> </span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:#0000BF;">Matei:</span><span style="color:black;"> Do you have any suggestion ?</span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:black;"> </span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:black;">Thank you</span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:black;">Emalayan</span></div></div><div><div class="yiv2016430077MsoNormal" style="background:white;"><span style="color:black;"> </span></div></div><div><div><div><div class="yiv2016430077MsoNormal" style="text-align:center;background:white;" align="center"><span style="font-size:10.0pt;font-family:"sans-serif";color:black;"><hr size="1" width="100%" align="center"></span></div><div
class="yiv2016430077MsoNormal" style="background:white;"><b><span style="font-size:10.0pt;font-family:"sans-serif";color:black;">From:</span></b><span style="font-size:10.0pt;font-family:"sans-serif";color:black;"> Michael Wilde <<a rel="nofollow" ymailto="mailto:wilde@mcs.anl.gov" target="_blank" href="mailto:wilde@mcs.anl.gov">wilde@mcs.anl.gov</a>><br><b>To:</b> Emalayan Vairavanathan <<a rel="nofollow" ymailto="mailto:svemalayan@yahoo.com" target="_blank" href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>> <br><b>Cc:</b> MosaStore <<a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>>; <a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a> <br><b>Sent:</b> Thursday, 29 March 2012 9:15 AM<br><b>Subject:</b> Re:
[Swift-devel] Data-aware scheduling in Swift ?</span><span style="color:black;"></span></div></div><div class="yiv2016430077MsoNormal" style="margin-bottom:12.0pt;background:white;"><span style="color:black;"><br>Swift will place an app() call on any free node. (As Jon just replied, while I was writing this...)<br><br>If we want to do an experiment with some kind of data affinity, we can try the following hack:<br><br>- Stage-A returns the node that it ran on<br>- swift script passes that as an arg "preferredNode(nodeName) to Stage-B<br>- scheduler tries to place Stage-B on the coaster named nodeName.<br><br>Its that last part thats the trickiest, as this will require a mod to the scheduler. And it gets trickier if the scheduler needs to try to defer Stage-B until nodeName can take a new job. It *might* be easier, in a first pass, to only place STage-B on nodeName if nodeName has a free job slot, else to place it anywhere.<br><br>But all of this
will require going into the coaster scheduler code.<br><br>I suggest we do this as a joint effort; I can try, with help from Mihael and Justin, to locate the code that we'd need to modify, if you are willing to do some experiments and hacking.<br><br>- Mike<br><br><br>----- Original Message -----<br>> From: "Emalayan Vairavanathan" <<a rel="nofollow" ymailto="mailto:svemalayan@yahoo.com" target="_blank" href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>><br>> To: <a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a><br>> Cc: "MosaStore" <<a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>><br>> Sent: Thursday, March 29, 2012 10:59:41 AM<br>> Subject: [Swift-devel] Data-aware scheduling in Swift ?<br>> Hi All,<br>>
<br>> <br>> I have a question about how swift schedules computations.<br>> <br>> <br>> Suppose there are two computation stages namely Stage-A and Stage-B in<br>> an application. Stage-A produces the data and Stage-B consumes the<br>> data . Could you please tell me how swift schedules these<br>> computations? Does it schedules Stage-A and Stage-B on the same node<br>> or on multiple nodes?<br>> Is it possible to configure the swift to schedules these computations<br>> on the same node (or is this the default behavior of swift ) ?<br>> <br>> <br>> <br>> <br>> Thank you<br>> Emalayan<br>> <br>> <br>> <br>> _______________________________________________<br>> Swift-devel mailing list<br>> <a rel="nofollow" ymailto="mailto:Swift-devel@ci.uchicago.edu" target="_blank" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>> <a rel="nofollow" target="_blank"
href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br><br>-- <br>Michael Wilde<br>Computation Institute, University of Chicago<br>Mathematics and Computer Science Division<br>Argonne National Laboratory<br><br>-- <br>You received this message because you are subscribed to the Google Groups "MosaStore" group.<br>To post to this group, send email to <a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>.<br>To unsubscribe from this group, send email to mosastore+<a rel="nofollow" ymailto="mailto:unsubscribe@googlegroups.com" target="_blank" href="mailto:unsubscribe@googlegroups.com">unsubscribe@googlegroups.com</a>.<br>For more options, visit this group at <a rel="nofollow" target="_blank"
href="http://groups.google.com/group/mosastore?hl=en">http://groups.google.com/group/mosastore?hl=en</a>.<br><br><br></span></div></div></div></div></div></div></div><br><br> </div> </div> </div></body></html>