[Swift-devel] Swift throttling

[Ben Clifford]

On Mon, 4 Feb 2008, Michael Wilde wrote:

> hopefully the slow-start sensing
> algorithms will sense that resource is already under strain and stay at a low
> submission rate.

I don't think it does that at all.

> - we modify the code to enable explicit limits on the algorithm to be set by
> the user, eg:
>  throttle.host.submitlimit - max # jobs that can be queued to a host
>  throttle.host.submitrate - max #jobs/sec that can be queued to a host
>                             (float)

parameters that control thsoe exist already, I think, for the whole 
workflow. In the single site case, site specific ones aren't needed 
because of that. If they were being implemented, it would probably be 
better to make them settable in the sites catalog so that they can be 
defined differently for each site.

throttle.scote.job.factor limits the number of concurrent jobs to 2 + 
100*throttle.score.job.factor   (so to achieve a limit of 52, set 
throttle.score.job.factor to 0.5)

There's a per-site profile setting:
maxSubmitRate - limits the maximum rate of job submission, in jobs per 
second.

--