[Swift-devel] swift-on-ec2

[Ioan Raicu]

Well, the dynamic provisioning assumes that Falkon is acquiring 
resources when it needs them.  This implies that it knows how to talk to 
the EC2 service, and it knows how to bootstrap a VM that has the 
necessary Falkon software stack.

I was actually hoping (at least in the short term) that static resource 
provisioning could be handled by the workspace service, talking to the 
EC2 service and bootstraping the VM (with the necesarry Falkon stack), 
and then once the Falkon executors register with the Falkon dispatcher, 
then Falkon handles the lightweight job management (in place of a 
traditional LRM). 

The provisioning to EC2 could be pushed onto Falkon in the future, but 
it is not currently on my immediate list of things to-do list.

Ioan

Kate Keahey wrote:
> Thanks Ben, this helps a lot! So it seems to me like we are talking 
> about combining dynamic provisioning with lightweight job management 
> which should be pluggable into swift.
>
> Ben Clifford wrote:
>> On Tue, 15 May 2007, Kate Keahey wrote:
>>
>>> As Ian says, Borja and I were planning to meet with Ioan on Thursday 
>>> to discuss interaction between Falkon and the workspace service (not 
>>> necessarily/exclusively in the EC2 context). I don't completely 
>>> understand the relationship between swift and falkon -- are there 
>>> specific applications or scenarios that you are trying to target in 
>>> this exercise?
>>
>> By virtue of the fact that they come from pretty much the same group 
>> of people, they're somewhat fuzzily related - but pretty much swift 
>> is generating (over the duration of its execution, rather than in one 
>> batch) a bunch of jobs that need executing (as well, as various 
>> things like file transfers). As it generates them, it sends them off 
>> to be executed. The official ways that are 'supported' by Swift are 
>> by executing them on the local machine and by sending them off 
>> through GRAM; however, people can plug in whatever they want to do 
>> submissions.
>>
>> I know less about Falkon because it isn't Swift, but the Falkon side 
>> of things is pretty much about running a bunch of jobs - it plugs 
>> into the abovementioned place in Swift so that Swift gives Falkon 
>> jobs to run, and Falkon runs them (with a goal of Falkon being, 
>> presumably, to run it much more efficiently than if they were 
>> submitted straight through GRAM - it seems to do pretty well).
>>
>> There's two things going on with swift - one is about making it 
>> straightforward to use at the low end of things, so that people can 
>> start using it easily - for the most part, that isn't interesting in 
>> itself; the other is about getting it to perform well at the high end 
>> of things, which is where the fun research is. Using Falkon and using 
>> EC2 are both on that side of things.
>>
>

-- 
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
       http://dsl.cs.uchicago.edu/
============================================
============================================