[Swift-devel] Re: swift performance optimization on BGP

[Ioan Raicu]

wow, 1 second overhead  ;)!  That's fantastic! Just out of curiosity, 
can you try a 1 rack run (16X more CPUs), with 8192 tasks? Will the 
overheads remain stable?

Ioan

Zhao Zhang wrote:
> that was the one I want to show the problem.
> The optimized results is here 
> http://www.ci.uchicago.edu/~zzhang/report-sleep-20081016-1808-96bsfgec/
>
> Total number of events: 512
> Shortest event (s): 30
> Longest event (s): 33
> Total duration of all events (s): 15865
> Mean event duration (s): 30.986328125
> Standard deviation of event duration (s): 0.784249612581342
> Maximum number of events at one time: 277
>
> zhao
>
> Ioan Raicu wrote:
>> Is this the right summary?
>>> Total number of events: 512
>>> Shortest event (s): 30
>>> Longest event (s): 43
>>> Total duration of all events (s): 17906
>>> Mean event duration (s): 34.97265625
>>> Standard deviation of event duration (s): 2.67205488703656
>>> Maximum number of events at one time: 262
>>>   
>> which shows that it takes ~35 sec for Swift to run a 30 sec task?  5 
>> sec overhead sounds pretty good, if we can keep it this low as we 
>> scale up to more CPUs :)
>>
>> Good job!
>>
>> Ioan
>>
>> Zhao Zhang wrote:
>>> Hi, All
>>>
>>> After the optimization last week, I was suffering the 2nd round task 
>>> poor performance, as you could tell
>>> from the "info trails" picture on 
>>> http://www.ci.uchicago.edu/~zzhang/report-sleep-20081016-1509-gqr8tea9/   
>>>
>>>
>>> with bunch of tests and verification, I found the poor performance 
>>> comes from the info log transfer. Thus I tried
>>> the collective IO for info log transfer, and it solves the problem, 
>>> I get such a beautiful picture:
>>>
>>> I am working more to integrate the Collective IO system along with 
>>> swift on BGP.
>>>
>>> zhao
>>>
>>> ------------------------------------------------------------------------ 
>>>
>>>
>>>
>>> ------------------------------------------------------------------------ 
>>>
>>>
>>
>> -- 
>> ===================================================
>> Ioan Raicu
>> Ph.D. Candidate
>> ===================================================
>> Distributed Systems Laboratory
>> Computer Science Department
>> University of Chicago
>> 1100 E. 58th Street, Ryerson Hall
>> Chicago, IL 60637
>> ===================================================
>> Email: iraicu at cs.uchicago.edu
>> Web:   http://www.cs.uchicago.edu/~iraicu
>> http://dev.globus.org/wiki/Incubator/Falkon
>> http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
>> ===================================================
>> ===================================================
>>
>>   
>

-- 
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
http://dev.globus.org/wiki/Incubator/Falkon
http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
===================================================
===================================================