[Swift-devel] Questions on coaster behavior

[Michael Wilde]

Hi Mihael, All,

I wanted to confirm some aspects of Coaster behavior that are still unclear to me after re-reading the UCC paper:

Scheduling: the coaster provider scheduler starts a number of worker blocks that are sized (in time and nodes) based on the size of its queue when it computes a schedule. This queue consists of jobs that were emitted by Swift to the provider based on the site throttle.

(note that by "job" here I mean the app() execution, not the LRM job).

But the coaster provider does not actually launch a job on a free coaster slot until the slot is available, right? Ie, there is no tight connection between the coaster slot that a job's time estimate contributed to, and the worker that the job is actually run on, right? Jobs are placed on workers at the last possible moment, and thus when a worker can take a job, it can get *any* job that is queued for that site. Is all this correct?  The key point behind this question being "is the coaster scheduler dynamic enough to hand cases where the job runtime estimates were conservative, and make best use of all available worker cores"? 

Staging: There are no cases in which the coaster provider staging mechanism pre-stages input data, right?

Thanks,

- Mike