[Swift-devel] Re: scheduler changes to deal with fast-failing sites

[Ben Clifford]

On Thu, 26 Jun 2008, lixi at uchicago.edu wrote:

> When you are not very busy, could you please explain the 
> details of the new improvement of scheduler for me?

Previous, a job would be submitted to a site if its load was less than its 
maximum load; maximum load was calculated as tscore * jobThrottle + 2. 
tscore ranges between 0 and 100 as score ranges between -inf and 
+inf. The maximum load will never go below 2, no matter how bad a site is.

In the new code, when score > 0 (the site is 'good') then the maximum load 
is calculated in a very similar way, as tscore * jobthrottle + 1. When 
score < 0 (the site is 'bad') then a different method is used. There will 
be delays between job submission. Two jobs will not be submitted to the 
same site within t ms of each other, with t = e^(-score) * 100.

So what should happen as a site is bad is that it will still sometimes be 
used, but the longer it remains bad, the longer the delay between 
attempted submissions will be.

--