[Swift-devel] Re: [Swft] Q about throttling

[Ian Foster]

Well, if there is some concern that throttling might be a problem, then 
trying a run with it turned off seems good.

I'm gathering from this exchange that this is not possible?

Ben Clifford wrote:
> But that isn't the base problem being investigated, right?
>
> On Thu, 21 Jun 2007, Ian Foster wrote:
>
>   
>> My original question was whether we could turn throttling off altogether. I'm
>> not sure if that was answered?
>>
>> Tiberiu Stef-Praun wrote:
>>     
>>> I did not look very deep into the throttling, mainly because I have to
>>> wait for my turn at using the Argonne cluster because of the large
>>> reservations that Ioan does for MolDyn
>>>
>>> Anyway, here is my experience (which Ian asked me to write down, but
>>> I'm still trying to improve on):
>>> - whatever one asks from Falkon, one seems to get, with the caveat
>>> that Falkon might release nodes when configured to look at an idle
>>> timer. In the case of the Econ workflow, I had 26 long running jobs,
>>> so I requested 30 nodes (which Falkon got for me)
>>> - there is a swift config file, $DVS_HOME/libexec/scheduler.xml in which I
>>> set
>>> <property name="jobThrottle" value="30"/>, but that seemed not to be
>>> enough to get all my 26 jobs running at the same time (as illustrated
>>> by the graphing of the Falkon log that Ioan showed me).
>>> - there are some other throttling parameters in
>>> $VDS_HOME/etc/swift.properties (which I also set to 30)
>>>
>>> The general observation is that I needed to modify the scheduler.xml
>>> config file, and I need to set larger throttle values that the limit
>>> of workers requested.
>>> In the current scheme (simply add Falkon to Swift as a provider) the
>>> Swift scheduler (the weighted site selection algorithm) adversely
>>> influences the optimal execution of the workflow.
>>> There might be other parameters to work with, but my opinion is that
>>> we should use a different (non-throttling) scheduler in combination
>>> with Falkon
>>>
>>> Tibi
>>>
>>> On 6/21/07, Mike Wilde <wilde at mcs.anl.gov> wrote:
>>>       
>>>> Ive had the same question - it seems that throttling is also problematic
>>>> for
>>>> Tibi in the econ workflow.
>>>>
>>>> Tibi, since you have looked pretty deeply into it, could you write up a
>>>> desription on how the algorithm works and how the parameters affect it.
>>>> Mihael, when you are back on central time next week, could you work with
>>>> TIbi on
>>>> this?  If its not already, this should be part of the Swift documentation.
>>>>
>>>> Then we should work on getting high-performance settings for the different
>>>> runtime environments we use, in particular Falkon as Ian asks.
>>>>
>>>> - Mike
>>>>
>>>>
>>>> Ian Foster wrote, On 6/21/2007 6:50 AM:
>>>>         
>>>>> Hi,
>>>>>
>>>>> I don't fully understand how throttling works in Swift/Karajan. However,
>>>>> I understand that even when using Falkon, we may be doing some
>>>>> throttling. Is there a reason to do that in this case, given that Falkon
>>>>> can maintain large numbers of tasks just fine?
>>>>>
>>>>> I ask this because in a recent MolDyn run, there seemed to be some
>>>>> uncertainty as to whether throttling was slowing down job dispatch. If
>>>>> we could turn it off altogether, that question would presumably go away.
>>>>>
>>>>> Ian.
>>>>>
>>>>>
>>>>>           
>>>> -- 
>>>> Mike Wilde
>>>> Computation Institute, University of Chicago
>>>> Math & Computer Science Division
>>>> Argonne National Laboratory
>>>> Argonne, IL   60439    USA
>>>> tel 630-252-7497 fax 630-252-1997
>>>>
>>>>         
>>>       
>>     
>
>   

-- 

   Ian Foster, Director, Computation Institute
Argonne National Laboratory & University of Chicago
Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439
Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637
Tel: +1 630 252 4619.  Web: www.ci.uchicago.edu.
      Globus Alliance: www.globus.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20070621/2d52ce1d/attachment.html>