[Swift-devel] Re: [Swft] Q about throttling

[Tiberiu Stef-Praun]

On 6/21/07, Ian Foster <foster at mcs.anl.gov> wrote:
> My original question was whether we could turn throttling off
> altogether. I'm not sure if that was answered?

I think the current answer is: just push way up the throttling limit.

>
> Tiberiu Stef-Praun wrote:
> > I did not look very deep into the throttling, mainly because I have to
> > wait for my turn at using the Argonne cluster because of the large
> > reservations that Ioan does for MolDyn
> >
> > Anyway, here is my experience (which Ian asked me to write down, but
> > I'm still trying to improve on):
> > - whatever one asks from Falkon, one seems to get, with the caveat
> > that Falkon might release nodes when configured to look at an idle
> > timer. In the case of the Econ workflow, I had 26 long running jobs,
> > so I requested 30 nodes (which Falkon got for me)
> > - there is a swift config file, $DVS_HOME/libexec/scheduler.xml in
> > which I set
> > <property name="jobThrottle" value="30"/>, but that seemed not to be
> > enough to get all my 26 jobs running at the same time (as illustrated
> > by the graphing of the Falkon log that Ioan showed me).
> > - there are some other throttling parameters in
> > $VDS_HOME/etc/swift.properties (which I also set to 30)
> >
> > The general observation is that I needed to modify the scheduler.xml
> > config file, and I need to set larger throttle values that the limit
> > of workers requested.
> > In the current scheme (simply add Falkon to Swift as a provider) the
> > Swift scheduler (the weighted site selection algorithm) adversely
> > influences the optimal execution of the workflow.
> > There might be other parameters to work with, but my opinion is that
> > we should use a different (non-throttling) scheduler in combination
> > with Falkon
> >
> > Tibi
> >
> > On 6/21/07, Mike Wilde <wilde at mcs.anl.gov> wrote:
> >> Ive had the same question - it seems that throttling is also
> >> problematic for
> >> Tibi in the econ workflow.
> >>
> >> Tibi, since you have looked pretty deeply into it, could you write up a
> >> desription on how the algorithm works and how the parameters affect it.
> >> Mihael, when you are back on central time next week, could you work
> >> with TIbi on
> >> this?  If its not already, this should be part of the Swift
> >> documentation.
> >>
> >> Then we should work on getting high-performance settings for the
> >> different
> >> runtime environments we use, in particular Falkon as Ian asks.
> >>
> >> - Mike
> >>
> >>
> >> Ian Foster wrote, On 6/21/2007 6:50 AM:
> >> > Hi,
> >> >
> >> > I don't fully understand how throttling works in Swift/Karajan.
> >> However,
> >> > I understand that even when using Falkon, we may be doing some
> >> > throttling. Is there a reason to do that in this case, given that
> >> Falkon
> >> > can maintain large numbers of tasks just fine?
> >> >
> >> > I ask this because in a recent MolDyn run, there seemed to be some
> >> > uncertainty as to whether throttling was slowing down job dispatch. If
> >> > we could turn it off altogether, that question would presumably go
> >> away.
> >> >
> >> > Ian.
> >> >
> >> >
> >>
> >> --
> >> Mike Wilde
> >> Computation Institute, University of Chicago
> >> Math & Computer Science Division
> >> Argonne National Laboratory
> >> Argonne, IL   60439    USA
> >> tel 630-252-7497 fax 630-252-1997
> >>
> >
> >
>
> --
>
>    Ian Foster, Director, Computation Institute
> Argonne National Laboratory & University of Chicago
> Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439
> Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637
> Tel: +1 630 252 4619.  Web: www.ci.uchicago.edu.
>       Globus Alliance: www.globus.org.
>
>


-- 
Tiberiu (Tibi) Stef-Praun, PhD
Research Staff, Computation Institute
5640 S. Ellis Ave, #405
University of Chicago
http://www-unix.mcs.anl.gov/~tiberius/