[Swift-devel] Re: Is there a way to make swift send 256 tasks at a time

[Ioan Raicu]

Michael Wilde wrote:
> ...
>
> But, I suspect you want *exactly* 256 to keep all the cores in each 
> pset busy without over-committing jobs. So what I think the best you 
> can do is start with the throttle at 3, and the score at 85 which will 
> give you 3 * 85 + 2 = 257 jobs. If the throttle creeps up for good 
> behavior, then you'll wind up with a little overcomiting, which isnt 
> bad. It wont go over 302 with a throttle of 3.
Can the throttle.score.job.factor be set to a float?  Then setting it to 
2.54 will give you exactly 256, assuming all tasks come back successful. 
>
> I suspect we could try a patch that prevents the score from changing.
>
> Can you tell from Falkon logs as we ramp up testing of this, whether 
> overcomiting is a problem? I suspect at worst it will exacerbate 
> tail-effects as the workflow is winding down and some resources sit 
> idle while others are overcomited.
Yes, it is the tail end of the runs that will be most affected.
>
> Lastly, note that the fact that the score can drop might eventually 
> prove useful in handling psets that go bad in the middle of long runs. 
Right! 

Ioan

-- 
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
http://dev.globus.org/wiki/Incubator/Falkon
http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
===================================================
===================================================