[Swift-devel] Re: [Swft] Q about throttling

[Ian Foster]

My original question was whether we could turn throttling off 
altogether. I'm not sure if that was answered?

Tiberiu Stef-Praun wrote:
> I did not look very deep into the throttling, mainly because I have to
> wait for my turn at using the Argonne cluster because of the large
> reservations that Ioan does for MolDyn
>
> Anyway, here is my experience (which Ian asked me to write down, but
> I'm still trying to improve on):
> - whatever one asks from Falkon, one seems to get, with the caveat
> that Falkon might release nodes when configured to look at an idle
> timer. In the case of the Econ workflow, I had 26 long running jobs,
> so I requested 30 nodes (which Falkon got for me)
> - there is a swift config file, $DVS_HOME/libexec/scheduler.xml in 
> which I set
> <property name="jobThrottle" value="30"/>, but that seemed not to be
> enough to get all my 26 jobs running at the same time (as illustrated
> by the graphing of the Falkon log that Ioan showed me).
> - there are some other throttling parameters in
> $VDS_HOME/etc/swift.properties (which I also set to 30)
>
> The general observation is that I needed to modify the scheduler.xml
> config file, and I need to set larger throttle values that the limit
> of workers requested.
> In the current scheme (simply add Falkon to Swift as a provider) the
> Swift scheduler (the weighted site selection algorithm) adversely
> influences the optimal execution of the workflow.
> There might be other parameters to work with, but my opinion is that
> we should use a different (non-throttling) scheduler in combination
> with Falkon
>
> Tibi
>
> On 6/21/07, Mike Wilde <wilde at mcs.anl.gov> wrote:
>> Ive had the same question - it seems that throttling is also 
>> problematic for
>> Tibi in the econ workflow.
>>
>> Tibi, since you have looked pretty deeply into it, could you write up a
>> desription on how the algorithm works and how the parameters affect it.
>> Mihael, when you are back on central time next week, could you work 
>> with TIbi on
>> this?  If its not already, this should be part of the Swift 
>> documentation.
>>
>> Then we should work on getting high-performance settings for the 
>> different
>> runtime environments we use, in particular Falkon as Ian asks.
>>
>> - Mike
>>
>>
>> Ian Foster wrote, On 6/21/2007 6:50 AM:
>> > Hi,
>> >
>> > I don't fully understand how throttling works in Swift/Karajan. 
>> However,
>> > I understand that even when using Falkon, we may be doing some
>> > throttling. Is there a reason to do that in this case, given that 
>> Falkon
>> > can maintain large numbers of tasks just fine?
>> >
>> > I ask this because in a recent MolDyn run, there seemed to be some
>> > uncertainty as to whether throttling was slowing down job dispatch. If
>> > we could turn it off altogether, that question would presumably go 
>> away.
>> >
>> > Ian.
>> >
>> >
>>
>> -- 
>> Mike Wilde
>> Computation Institute, University of Chicago
>> Math & Computer Science Division
>> Argonne National Laboratory
>> Argonne, IL   60439    USA
>> tel 630-252-7497 fax 630-252-1997
>>
>
>

-- 

   Ian Foster, Director, Computation Institute
Argonne National Laboratory & University of Chicago
Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439
Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637
Tel: +1 630 252 4619.  Web: www.ci.uchicago.edu.
      Globus Alliance: www.globus.org.