[Swift-devel] swift-on-ec2

[Ioan Raicu]

Tim Freeman wrote:
> On Wed, 16 May 2007 11:55:18 -0500
> Ioan Raicu <iraicu at cs.uchicago.edu> wrote:
>
>   
>> Hi,
>> I am just catching up with emails from last night...
>>
>> Ben Clifford wrote:
>>     
>>> On Tue, 15 May 2007, Kate Keahey wrote:
>>>
>>>   
>>>       
>>>> As Ian says, Borja and I were planning to meet with Ioan on Thursday to 
>>>> discuss interaction between Falkon and the workspace service (not 
>>>> necessarily/exclusively in the EC2 context). I don't completely 
>>>> understand the relationship between swift and falkon -- are there 
>>>> specific applications or scenarios that you are trying to target in this 
>>>> exercise?
>>>>     
>>>>         
>>> By virtue of the fact that they come from pretty much the same group of 
>>> people, they're somewhat fuzzily related - but pretty much swift is 
>>> generating (over the duration of its execution, rather than in one batch) 
>>> a bunch of jobs that need executing (as well, as various things like file 
>>> transfers). As it generates them, it sends them off to be executed. The 
>>> official ways that are 'supported' by Swift are by executing them on the 
>>> local machine and by sending them off through GRAM; however, people can 
>>> plug in whatever they want to do submissions.
>>>
>>> I know less about Falkon because it isn't Swift, but the Falkon side of 
>>> things is pretty much about running a bunch of jobs - it plugs into the 
>>> abovementioned place in Swift so that Swift gives Falkon jobs to run, and 
>>> Falkon runs them (with a goal of Falkon being, presumably, to run it much 
>>> more efficiently than if they were submitted straight through GRAM - it 
>>> seems to do pretty well).
>>>   
>>>       
>> We intentionally made Falkon's interface and semantics as similar as 
>> possible to that of GRAM, so applications that normally used GRAM could 
>> easily change to Falkon.
>>     
>>> There's two things going on with swift - one is about making it 
>>> straightforward to use at the low end of things, so that people can start 
>>> using it easily - for the most part, that isn't interesting in itself; the 
>>> other is about getting it to perform well at the high end of things, which 
>>> is where the fun research is. Using Falkon and using EC2 are both on that 
>>> side of things.
>>>   
>>>       
>> Right! 
>>
>> Falkon is certainly about getting more performance from the same hardware. 
>>
>> EC2 on the other hand is more about a new paradigm of how resources are 
>> acquired.  In the batch-scheduled world, the demand for resources is 
>> usually higher than the supply.  In EC2, its likely that the supply for 
>> resources is higher than the demand.  With that said, it means that with 
>> EC2, it is likely that you could always get more resources now if you 
>> were willing to pay for them
>>     
>
> That's not entirely true at this particular point in time:
>
> http://www.pcworld.com/article/id,130832-c,webservices/article.html
>
> "We hate being capacity-constrained," Bezos said. "It's not the right way to
> run a business. We are trying to get ourselves in a position with EC2 where we
> will be demand-constrained instead of capacity-constrained."
>
>   
But this doesn't make much sense.  I think these guys get $700 or so a 
year for each VM they run, that means that they are charging more money 
over the lifetime of the machine than it costs to purchase and maintain 
the machine (assuming they are cheap computers).  With this said, it 
seems that they should be adding more resources as the demand grows, so 
they always have resources available if someone asks for them... at 
least that is what I am expecting from such as service.  If this is not 
the case now, I hope it will be in the future!

Ioan
>   
>> ... this could have implications on the 
>> resource allocation and management policies that govern when it makes 
>> sense to get more resources and when not to.
>>     
>
> Right now for example, we're programming a little feature into the workspace-EC2
> gateway that limits the amount of money an entity can spend :-) 
>
> Tim 
>
>
>   
>>   Using EC2 might be about 
>> performance, but the really interesting part that I see emerging is a 
>> new model that deviates from the traditional batch-scheduled systems the 
>> Grid community has grown accustomed to.
>>
>> Ioan
>>     
>
>
>   

-- 
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
       http://dsl.cs.uchicago.edu/
============================================
============================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20070516/64720490/attachment.html>