[Swift-user] Please clarify throttle parameters for coasters and GT2 issues

[Allan Espinosa]

In pushing Ranger's current scheduling policies of 50 SGE jobs, we can
push a max of 800 cpus.  I have tried this before using the gt2
interface.

-Allan

On Thu, Apr 2, 2009 at 8:52 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
> On Thu, 2009-04-02 at 21:20 -0500, Michael Wilde wrote:
>> I (and colleagues Im working with) have a few related questions:
>>
>> At some point guidelines were posted regarding "safe" throttle values
>> for sending to GT2 GRAM sites. I recall "max 40 jobs". Can you clarify
>> if that number is still the best practice,
>
> Yes. GT2 hasn't changed much since.
>
>>  and how to the 4-5 throttle
>> parameters to conform?
>
> The defaults are pretty much geared towards the gt2/gridftp combo.
>
>>
>> Then, do those same values apply to coasters?
>
> No. Throttles in the range of 32-256 (or maybe even more) are not
> unreasonable with coasters.
>
>>
>> Finally, with the recent successes in high-volume coaster runs on Ranger
>> - 190K jobs in ~ 5 hours, and even the earlier runs of 65K jobs - how
>> were those achieved without tripping into the GRAM overhead limits,
>> given that Ranger as far as I know has only GT2 GRAM and even submitting
>> locallly to SGE must go through GRAM since we have no direct SGE
>> provider?  Are the "safe" limits for Ranger simply higer,or is there
>> something else involved that makes this practical?  In other words,
>> please share and post how to get lots of jobs through Ranger.
>
> coasterWorkersPerNode=16. That gives you 640 cpus with exactly 40 gram
> jobs.
>




-- 
Allan M. Espinosa <http://allan.88-mph.net/blog>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>