[Swift-devel] virtual clusters an swift

[Mihael Hategan]

On Wed, 2011-05-11 at 09:19 +0000, Ben Clifford wrote:
> > One, which Mike mentioned, is a glide-in like thing (called coasters),
> > but the main purpose of that is improved performance (e.g. faster job
> > submission rates, avoiding shared FSs as much as possible). On the other
> > hand, if there are non-trivial costs associated with allocating nodes,
> > that might be worth considering.
> 
> This sounds like you see two ways of doing things: 
> one-vm-per-swift-app-call and one-cluster-of-vms-running-coasters

Mostly. I was thinking of a BGP like configuration in which each job
allocates and boots a set of worker nodes. What I was trying to say in
my own twisted way was that coasters, while providing an integrated and
fast solution, are not fundamentally needed. A plain PBS scheme could
work as well, and if the jobs aren't too small, various costs may get
amortized.

> 
> Whenever I thought about this before, something like coasters (or 
> appropriately configured PBS when I first thought about it), seemed the 
> sensible of the two.

Right.

> 
> I think node alloc time (assuming it behaves like EC2 stuff i've done 
> elsewhere) is on the order of booting a machine from scratch.
> 

And right. Same as BGP. Those are environments where you may want to
grab nodes and stick to them.