[Swift-user] Throttling within Swift

[Michael Wilde]

> From: "Lorenzo Pesce" <lpesce at uchicago.edu>

> So, if I want to run 24 jobs per node on a 24 core node and use 100
> nodes, I set it to 24?

Yes. 24 would enable 2400 jobs to get submitted to this site at once.

Thats what you would need if for example you set your sites.xml parameters for a coasters-enabled site to maxnodes 100 slote 1, or slots 100 maxnodes 1, or any similar factoring, with jobsPerNode 24.

There are three levels of throttling you need to be aware of (and can use): the two above, plus foreach.maxthreads, which limits each individual foreach.

- Mike

> 
> On Jun 6, 2012, at 5:29 PM, Michael Wilde wrote:
> 
> > Scott, its best to adjust the throttle using the sites file. Use
> > these two xml elements in your <pool> element:
> >
> >    <profile namespace="karajan" key="jobThrottle">.63</profile>
> >    <profile namespace="karajan" key="initialScore">10000</profile>
> >
> > Always set initialScore to 10000.
> > Set the actual throttle by setting jobThrottle to (N-1)/100 to run N
> > jobs at a time.
> >
> > So for N=2,4,8,16 set jobThrottle to .01, .03, .07, and .15
> >
> > Thats admittedly rather arcane, but works that way for historical
> > reasons. We hope to simplify this in the future.
> >
> > - Mike
> >
> >
> >
> > ----- Original Message -----
> >> From: "Scott Krieder" <skrieder at iit.edu>
> >> To: swift-user at ci.uchicago.edu
> >> Sent: Wednesday, June 6, 2012 5:07:03 PM
> >> Subject: [Swift-user] Throttling within Swift
> >> Quick question about throttling within Swift:
> >>
> >>
> >> I have a swift script that launches 64 binary files in parallel
> >> within
> >> a foreach loop.
> >>
> >>
> >> I would like Swift to throttle so that there are only 2 binaries
> >> running at any given time. Then I would like to change the throttle
> >> for 4, 8, and 16.
> >>
> >>
> >> I found several different layers at which I can change the
> >> throttle.
> >> For the above configuration should I change the throttle in my
> >> sites.xml or do I need to update the swift.properties file?
> >>
> >>
> >> Thank you,
> >> Scott
> >>
> >>
> >> Scott Krieder
> >> DataSys Laboratory
> >> Illinois Institute of Technology
> >> _______________________________________________
> >> Swift-user mailing list
> >> Swift-user at ci.uchicago.edu
> >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> >
> > --
> > Michael Wilde
> > Computation Institute, University of Chicago
> > Mathematics and Computer Science Division
> > Argonne National Laboratory
> >
> > _______________________________________________
> > Swift-user mailing list
> > Swift-user at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory