[Swift-devel] coaster status summary

[Michael Wilde]

Mihael, this is great progress - very exciting.
Some questions (dont need answers right away):

How would the end user use it? Manually start a service?
Is the service a separate process, or in the swift jvm?
How are the number of workers set or adjusted?
Does a service manage workers on one cluster or many?
At 180 jobs/sec with 10 workers, what were the CPU loads on swift, 
worker and service?

Do you want to try this on the workflows we're running on Falkon on the 
BGP and SiCortex?

Im eager to try it when you feel its ready for others to test.

Nice work!

- Mike



On 4/4/08 4:39 AM, Mihael Hategan wrote:
> I've been asked for a summary of the status of the coaster prototype, so
> here it is:
> - It's a prototype so bugs are plenty
> - It's self deployed (you don't need to start a service on the target
> cluster)
> - You can also use it while starting a service on the target cluster
> - There is a worker written in Perl
> - It uses encryption between client and coaster service
> - It uses UDP between the service and the workers (this may prove to be
> better or worse choice than TCP)
> - A preliminary test done locally shows an amortized throughput of
> around 180 jobs/s (/bin/date). This was done with encryption and with 10
> workers. Pretty picture attached (total time vs. # of jobs)
> 
> To do:
> - The scheduling algorithm in the service needs a bit more work
> - When worker messages are lost, some jobs may get lost (i.e. needs more
> fault tolerance)
> - Start testing it on actual clusters
> - Do some memory consumption benchmarks
> - Better allocation strategy for workers
> 
> Mihael
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel