[Swift-devel] throttling parameters with coasters

[Mihael Hategan]

On Tue, 2009-02-24 at 18:52 -0600, Allan Espinosa wrote:
> In coasters, are throtte.submit, throttle.host.submit,
> throttle.job.factor parameters ignored ?
> 
> Looking on how swift submits initial requests, it seems that
> throttle.job.factor affects the number of coaster nodes it will submit
> to the LRM. If I have
>   <profile namespace="karajan" key="initialScore">1</profile>
>     <profile namespace="karajan" key="jobThrottle">1</profile>
> Swift spawns 4 coasters.  4*16=64 processors available to me.  I
> observe that throughout the job this number did not increase

A job throttle of 1 pretty much caps the total number of concurrent jobs
at 100.

> 
> Next, in my swift.properties, I have
> throttle.submit=4
> throttle.host.submit=2
> 
> But in the runtime,
> 
> rogress:  Selecting site:2809 Submitting:17 Active:40 Stage out:31
> Finished successfully:103
> Progress:  Selecting site:2809 Submitting:17 Active:40 Stage out:30 Finished su
> 
> so the 2 parameters does not apply to coaster submissions?

The "submitting" printed by the progress ticker is not the same as the
"submit" in swift.properties.

>From a cog abstractions perspective, 4 concurrent submissions means that
only 4 calls to TaskHandler.submit(Task) can be active at one time.

>From swift's perspective it means that the job was queued to the
scheduler and awaits its turn to be one of the 4 to go through
TaskHandler.submit().