[Swift-devel] coaster status summary

[Ioan Raicu]

I meant to send this before, but somehow it seems to have gotten stuck 
in my draft folder ;(

We are running out of time on the papers we are writing now, but it 
would certainly been a good comparison of implementations, assumptions, 
trade-offs, performance, etc... for a future paper!

I am eager to learn more about it!

Ioan

Michael Wilde wrote:
> Mihael, this is great progress - very exciting.
> Some questions (dont need answers right away):
>
> How would the end user use it? Manually start a service?
> Is the service a separate process, or in the swift jvm?
> How are the number of workers set or adjusted?
> Does a service manage workers on one cluster or many?
> At 180 jobs/sec with 10 workers, what were the CPU loads on swift, 
> worker and service?
>
> Do you want to try this on the workflows we're running on Falkon on 
> the BGP and SiCortex?
>
> Im eager to try it when you feel its ready for others to test.
>
> Nice work!
>
> - Mike
>
>
>
> On 4/4/08 4:39 AM, Mihael Hategan wrote:
>> I've been asked for a summary of the status of the coaster prototype, so
>> here it is:
>> - It's a prototype so bugs are plenty
>> - It's self deployed (you don't need to start a service on the target
>> cluster)
>> - You can also use it while starting a service on the target cluster
>> - There is a worker written in Perl
>> - It uses encryption between client and coaster service
>> - It uses UDP between the service and the workers (this may prove to be
>> better or worse choice than TCP)
>> - A preliminary test done locally shows an amortized throughput of
>> around 180 jobs/s (/bin/date). This was done with encryption and with 10
>> workers. Pretty picture attached (total time vs. # of jobs)
>>
>> To do:
>> - The scheduling algorithm in the service needs a bit more work
>> - When worker messages are lost, some jobs may get lost (i.e. needs more
>> fault tolerance)
>> - Start testing it on actual clusters
>> - Do some memory consumption benchmarks
>> - Better allocation strategy for workers
>>
>> Mihael
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>

-- 
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
http://dev.globus.org/wiki/Incubator/Falkon
http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
===================================================
===================================================