[Swift-devel] virtual clusters an swift

[Mihael Hategan]

On Tue, 2011-05-10 at 10:38 -1000, John Bresnahan wrote:
> Hello all,
> 
> I am looking to have make a repeatable launch program that brings up virtual clusters across cloud 
> domains for use with swift.  I have a few questions about what swift assumes about a cluster on 
> which it runs.
> 
> How does it access the machines?  ssh?

There are various "providers" that implement how swift accesses a
machine. SSH is one of them. GT is another. Those are the remote ones.
It can also talk directly to local schedulers, but that's probably not
what you want.

>   Is any other base software needed or assumed to be running?

There is an assumption of some kind of shared filesystem.
Other than that, it depends. There are various schemes to get things
running.

One, which Mike mentioned, is a glide-in like thing (called coasters),
but the main purpose of that is improved performance (e.g. faster job
submission rates, avoiding shared FSs as much as possible). On the other
hand, if there are non-trivial costs associated with allocating nodes,
that might be worth considering.

The shared filesystem requirement can probably be circumvented if
coasters are used for file staging. In that case, the "worker" script
somehow needs to be made available to the worker nodes.

> Can it deal with clusters that have a single head node with an external IP but all internal IPs for 
> worker nodes?

In most cases that's what we assumed to be the case, so yes. When not
using coasters, that's pretty much there by default, because all
communication is with head node services. When using coasters, requests
are piped through the head node which acts as a bridge between WNs and
the client.

> Is there anything else that is assumed about the machines that swift uses.

/bin/bash and standard tools (cp, mv, etc.) on both head node and worker
nodes.

Coasters also require md5sum or gmd5sum, wget or curl, and java, all on
the head node and perl on the worker nodes. There are ways to get around
md5 and wget.

With coasters, a valid GSI proxy is needed (as well as the proper CA
certs on both client and head node). Sadly, that is even in the case
when SSH is used.

Mihael