[Swift-devel] Re: [Swft] Q about throttling

[Ioan Raicu]

I think Ben is right, in this particular instance, Swift submitted all 
26 jobs, and Falkon dispatched 24 of them, and held 2 of them in the 
wait queue.  Throttling was not the issue here.  At first glance, I 
would say that although you asked for 30 nodes at the beining, you might 
have lost some due to idle time limit being reached, and hence when you 
started the 26 jobs, you only had 24 executors ready.  Can you send me 
these two logs: service/logs/GenericPortalWS_perf_per_sec.log, and 
service/logs/GenericPortalWS_taskPerf.log and I will try to superimpose 
the # of busy and free executors on top of the graph you sent out 
showing the per task information.

Ioan

Ben Clifford wrote:
> My interpretation of the graph is:
>
> The two jobs that didn't get run till later (the 'spare' jobs) are 
> submitted into falkon at approx t=0, along with the 24 'run straight away' 
> jobs.
>
> Swift isn't holding them back.
>
> Falkon indicates that it is aware of them from approx time = 0 but doesn't 
> run them until t=500000.
>
> That means, I think, that they're getting into Falkons queue right at the 
> start, and its something happening with how Falkon places them onto worker 
> nodes that isn't right here.
>
> At least that's my first impression.
>
> On Thu, 21 Jun 2007, Tiberiu Stef-Praun wrote:
>
>   
>> No
>> I'm saying that swift throttle value will allow me to make full use of
>> all the nodes that Falkon makes available for me. I know that I had 26
>> jobs to be run, and I requested (and had) 30 nodes in the cluster.
>> Somehow only 24 jobs run in the first time, so I'm going to push up
>> the throttle value in Swift
>>
>> Tibi
>>
>>
>> On 6/21/07, Ben Clifford <benc at hawaga.org.uk> wrote:
>>     
>>>
>>> On Thu, 21 Jun 2007, Tiberiu Stef-Praun wrote:
>>>
>>>       
>>>> No
>>>> My chart shows that if I had two more machines during the first stage
>>>> run (the first 26 jobs), I would have avoided a long wait (50000 ms ,
>>>> or about 9 minutes) for the last two jobs from the first batch to
>>>> finish.
>>>> This is why I need to redo the Econ run, with a different throttle
>>>> value for Swift.
>>>>         
>>> So you are saying that changing the 'throttle value for swift' will
>>> allocate more machines for you?
>>>
>>> --
>>>
>>>       
>>
>>     
>
>   

-- 
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
       http://dsl.cs.uchicago.edu/
============================================
============================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20070621/dc09bec6/attachment.html>