[Swift-devel] Data-aware scheduling in Swift ?

[Michael Wilde]

Swift will place an app() call on any free node. (As Jon just replied, while I was writing this...)

If we want to do an experiment with some kind of data affinity, we can try the following hack:

- Stage-A returns the node that it ran on
- swift script passes that as an arg "preferredNode(nodeName) to Stage-B
- scheduler tries to place Stage-B on the coaster named nodeName.

Its that last part thats the trickiest, as this will require a mod to the scheduler. And it gets trickier if the scheduler needs to try to defer Stage-B until nodeName can take a new job.  It *might* be easier, in a first pass, to only place STage-B on nodeName if nodeName has a free job slot, else to place it anywhere.

But all of this will require going into the coaster scheduler code.

I suggest we do this as a joint effort; I can try, with help from Mihael and Justin, to locate the code that we'd need to modify, if you are willing to do some experiments and hacking.

- Mike


----- Original Message -----
> From: "Emalayan Vairavanathan" <svemalayan at yahoo.com>
> To: swift-devel at ci.uchicago.edu
> Cc: "MosaStore" <mosastore at googlegroups.com>
> Sent: Thursday, March 29, 2012 10:59:41 AM
> Subject: [Swift-devel] Data-aware scheduling in Swift ?
> Hi All,
> 
> 
> I have a question about how swift schedules computations.
> 
> 
> Suppose there are two computation stages namely Stage-A and Stage-B in
> an application. Stage-A produces the data and Stage-B consumes the
> data . Could you please tell me how swift schedules these
> computations? Does it schedules Stage-A and Stage-B on the same node
> or on multiple nodes?
> Is it possible to configure the swift to schedules these computations
> on the same node (or is this the default behavior of swift ) ?
> 
> 
> 
> 
> Thank you
> Emalayan
> 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory