[Swift-devel] virtual clusters an swift

[Michael Wilde]

Hi John,

For cloud use, Swift uses a resource-provisioning mechanism called "coasters" which is introduced here:

  http://www.ci.uchicago.edu/swift/guides/userguide.php#coasters

and

  http://wiki.cogkit.org/wiki/Coasters

This short paper from the recent CCA-11 workshop explains a bit more on the cloud configuration:

  http://cca11.org/files/2011/04/p16.pdf

To configure for a cloud: on every cloud worker node, start a coaster "worker". This is is Perl script, worker.pl, located in the Swift bin/ dir. Command line args tell worker.pl to connect either to a Swift client, or to an intermediary Swift coaster-service.  Using the coaster-service will allow you to create a long-lived Swift coaster pool that can support the repeated execution of Swift scripts by Swift client tasks.

We'll locate and package up for you a simple set of startup scripts that can be run on any collection of hosts that you can ssh to, as well as the scripts that we used on Bionimbus (which used ssh port forwarding).

- Mike






----- Original Message -----
> Hello all,
> 
> I am looking to have make a repeatable launch program that brings up
> virtual clusters across cloud
> domains for use with swift. I have a few questions about what swift
> assumes about a cluster on
> which it runs.
> 
> How does it access the machines? ssh? Is any other base software
> needed or assumed to be running?
> Can it deal with clusters that have a single head node with an
> external IP but all internal IPs for
> worker nodes?
> Is there anything else that is assumed about the machines that swift
> uses.
> 
> Thanks,
> 
> John
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory