[Swift-devel] swift-on-ec2

[Yong Zhao]

I'd think the the workspace manager should be able to do that, and not
statically, but allocate new virtual nodes as requested.

Yong.

On Wed, 16 May 2007, Ioan Raicu wrote:

> Well, the dynamic provisioning assumes that Falkon is acquiring
> resources when it needs them.  This implies that it knows how to talk to
> the EC2 service, and it knows how to bootstrap a VM that has the
> necessary Falkon software stack.
>
> I was actually hoping (at least in the short term) that static resource
> provisioning could be handled by the workspace service, talking to the
> EC2 service and bootstraping the VM (with the necesarry Falkon stack),
> and then once the Falkon executors register with the Falkon dispatcher,
> then Falkon handles the lightweight job management (in place of a
> traditional LRM).
>
> The provisioning to EC2 could be pushed onto Falkon in the future, but
> it is not currently on my immediate list of things to-do list.
>
> Ioan
>
> Kate Keahey wrote:
> > Thanks Ben, this helps a lot! So it seems to me like we are talking
> > about combining dynamic provisioning with lightweight job management
> > which should be pluggable into swift.
> >
> > Ben Clifford wrote:
> >> On Tue, 15 May 2007, Kate Keahey wrote:
> >>
> >>> As Ian says, Borja and I were planning to meet with Ioan on Thursday
> >>> to discuss interaction between Falkon and the workspace service (not
> >>> necessarily/exclusively in the EC2 context). I don't completely
> >>> understand the relationship between swift and falkon -- are there
> >>> specific applications or scenarios that you are trying to target in
> >>> this exercise?
> >>
> >> By virtue of the fact that they come from pretty much the same group
> >> of people, they're somewhat fuzzily related - but pretty much swift
> >> is generating (over the duration of its execution, rather than in one
> >> batch) a bunch of jobs that need executing (as well, as various
> >> things like file transfers). As it generates them, it sends them off
> >> to be executed. The official ways that are 'supported' by Swift are
> >> by executing them on the local machine and by sending them off
> >> through GRAM; however, people can plug in whatever they want to do
> >> submissions.
> >>
> >> I know less about Falkon because it isn't Swift, but the Falkon side
> >> of things is pretty much about running a bunch of jobs - it plugs
> >> into the abovementioned place in Swift so that Swift gives Falkon
> >> jobs to run, and Falkon runs them (with a goal of Falkon being,
> >> presumably, to run it much more efficiently than if they were
> >> submitted straight through GRAM - it seems to do pretty well).
> >>
> >> There's two things going on with swift - one is about making it
> >> straightforward to use at the low end of things, so that people can
> >> start using it easily - for the most part, that isn't interesting in
> >> itself; the other is about getting it to perform well at the high end
> >> of things, which is where the fun research is. Using Falkon and using
> >> EC2 are both on that side of things.
> >>
> >
>
> --
> ============================================
> Ioan Raicu
> Ph.D. Student
> ============================================
> Distributed Systems Laboratory
> Computer Science Department
> University of Chicago
> 1100 E. 58th Street, Ryerson Hall
> Chicago, IL 60637
> ============================================
> Email: iraicu at cs.uchicago.edu
> Web:   http://www.cs.uchicago.edu/~iraicu
>        http://dsl.cs.uchicago.edu/
> ============================================
> ============================================
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>