[Swift-user] Please clarify throttle parameters for coasters and GT2 issues

[Michael Wilde]

Allan,

50 SGE jobs is Ranger's queue limit, but its above the 40-job "safe" 
limit suggested by the Swift developers. Above 40 we risk causing sever 
overhead on their gatekeeper, which I think doubles as one of the login 
hosts.

So I would urge you that when using GT2 to submit (with and without 
coasters) that you stay under 40, until coasters can create more workers 
with less jobs.

- Mike


On 4/3/09 11:58 AM, Allan Espinosa wrote:
> In pushing Ranger's current scheduling policies of 50 SGE jobs, we can
> push a max of 800 cpus.  I have tried this before using the gt2
> interface.
> 
> -Allan
> 
> On Thu, Apr 2, 2009 at 8:52 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
>> On Thu, 2009-04-02 at 21:20 -0500, Michael Wilde wrote:
>>> I (and colleagues Im working with) have a few related questions:
>>>
>>> At some point guidelines were posted regarding "safe" throttle values
>>> for sending to GT2 GRAM sites. I recall "max 40 jobs". Can you clarify
>>> if that number is still the best practice,
>> Yes. GT2 hasn't changed much since.
>>
>>>  and how to the 4-5 throttle
>>> parameters to conform?
>> The defaults are pretty much geared towards the gt2/gridftp combo.
>>
>>> Then, do those same values apply to coasters?
>> No. Throttles in the range of 32-256 (or maybe even more) are not
>> unreasonable with coasters.
>>
>>> Finally, with the recent successes in high-volume coaster runs on Ranger
>>> - 190K jobs in ~ 5 hours, and even the earlier runs of 65K jobs - how
>>> were those achieved without tripping into the GRAM overhead limits,
>>> given that Ranger as far as I know has only GT2 GRAM and even submitting
>>> locallly to SGE must go through GRAM since we have no direct SGE
>>> provider?  Are the "safe" limits for Ranger simply higer,or is there
>>> something else involved that makes this practical?  In other words,
>>> please share and post how to get lots of jobs through Ranger.
>> coasterWorkersPerNode=16. That gives you 640 cpus with exactly 40 gram
>> jobs.
>>
> 
> 
> 
>