[Swift-devel] Re: [Swft] Q about throttling

[Ben Clifford]

But that isn't the base problem being investigated, right?

On Thu, 21 Jun 2007, Ian Foster wrote:

> My original question was whether we could turn throttling off altogether. I'm
> not sure if that was answered?
> 
> Tiberiu Stef-Praun wrote:
> > I did not look very deep into the throttling, mainly because I have to
> > wait for my turn at using the Argonne cluster because of the large
> > reservations that Ioan does for MolDyn
> > 
> > Anyway, here is my experience (which Ian asked me to write down, but
> > I'm still trying to improve on):
> > - whatever one asks from Falkon, one seems to get, with the caveat
> > that Falkon might release nodes when configured to look at an idle
> > timer. In the case of the Econ workflow, I had 26 long running jobs,
> > so I requested 30 nodes (which Falkon got for me)
> > - there is a swift config file, $DVS_HOME/libexec/scheduler.xml in which I
> > set
> > <property name="jobThrottle" value="30"/>, but that seemed not to be
> > enough to get all my 26 jobs running at the same time (as illustrated
> > by the graphing of the Falkon log that Ioan showed me).
> > - there are some other throttling parameters in
> > $VDS_HOME/etc/swift.properties (which I also set to 30)
> > 
> > The general observation is that I needed to modify the scheduler.xml
> > config file, and I need to set larger throttle values that the limit
> > of workers requested.
> > In the current scheme (simply add Falkon to Swift as a provider) the
> > Swift scheduler (the weighted site selection algorithm) adversely
> > influences the optimal execution of the workflow.
> > There might be other parameters to work with, but my opinion is that
> > we should use a different (non-throttling) scheduler in combination
> > with Falkon
> > 
> > Tibi
> > 
> > On 6/21/07, Mike Wilde <wilde at mcs.anl.gov> wrote:
> > > Ive had the same question - it seems that throttling is also problematic
> > > for
> > > Tibi in the econ workflow.
> > > 
> > > Tibi, since you have looked pretty deeply into it, could you write up a
> > > desription on how the algorithm works and how the parameters affect it.
> > > Mihael, when you are back on central time next week, could you work with
> > > TIbi on
> > > this?  If its not already, this should be part of the Swift documentation.
> > > 
> > > Then we should work on getting high-performance settings for the different
> > > runtime environments we use, in particular Falkon as Ian asks.
> > > 
> > > - Mike
> > > 
> > > 
> > > Ian Foster wrote, On 6/21/2007 6:50 AM:
> > > > Hi,
> > > >
> > > > I don't fully understand how throttling works in Swift/Karajan. However,
> > > > I understand that even when using Falkon, we may be doing some
> > > > throttling. Is there a reason to do that in this case, given that Falkon
> > > > can maintain large numbers of tasks just fine?
> > > >
> > > > I ask this because in a recent MolDyn run, there seemed to be some
> > > > uncertainty as to whether throttling was slowing down job dispatch. If
> > > > we could turn it off altogether, that question would presumably go away.
> > > >
> > > > Ian.
> > > >
> > > >
> > > 
> > > -- 
> > > Mike Wilde
> > > Computation Institute, University of Chicago
> > > Math & Computer Science Division
> > > Argonne National Laboratory
> > > Argonne, IL   60439    USA
> > > tel 630-252-7497 fax 630-252-1997
> > > 
> > 
> > 
> 
>