[Swift-devel] Re: swift performance optimization on BGP

[Zhao Zhang]

yep, I will try that after I got ready for Mike's presentation next Wed.

zhao

Ioan Raicu wrote:
> wow, 1 second overhead  ;)!  That's fantastic! Just out of curiosity, 
> can you try a 1 rack run (16X more CPUs), with 8192 tasks? Will the 
> overheads remain stable?
>
> Ioan
>
> Zhao Zhang wrote:
>> that was the one I want to show the problem.
>> The optimized results is here 
>> http://www.ci.uchicago.edu/~zzhang/report-sleep-20081016-1808-96bsfgec/
>>
>> Total number of events: 512
>> Shortest event (s): 30
>> Longest event (s): 33
>> Total duration of all events (s): 15865
>> Mean event duration (s): 30.986328125
>> Standard deviation of event duration (s): 0.784249612581342
>> Maximum number of events at one time: 277
>>
>> zhao
>>
>> Ioan Raicu wrote:
>>> Is this the right summary?
>>>> Total number of events: 512
>>>> Shortest event (s): 30
>>>> Longest event (s): 43
>>>> Total duration of all events (s): 17906
>>>> Mean event duration (s): 34.97265625
>>>> Standard deviation of event duration (s): 2.67205488703656
>>>> Maximum number of events at one time: 262
>>>>   
>>> which shows that it takes ~35 sec for Swift to run a 30 sec task?  5 
>>> sec overhead sounds pretty good, if we can keep it this low as we 
>>> scale up to more CPUs :)
>>>
>>> Good job!
>>>
>>> Ioan
>>>
>>> Zhao Zhang wrote:
>>>> Hi, All
>>>>
>>>> After the optimization last week, I was suffering the 2nd round 
>>>> task poor performance, as you could tell
>>>> from the "info trails" picture on 
>>>> http://www.ci.uchicago.edu/~zzhang/report-sleep-20081016-1509-gqr8tea9/   
>>>>
>>>>
>>>> with bunch of tests and verification, I found the poor performance 
>>>> comes from the info log transfer. Thus I tried
>>>> the collective IO for info log transfer, and it solves the problem, 
>>>> I get such a beautiful picture:
>>>>
>>>> I am working more to integrate the Collective IO system along with 
>>>> swift on BGP.
>>>>
>>>> zhao
>>>>
>>>> ------------------------------------------------------------------------ 
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------ 
>>>>
>>>>
>>>
>>> -- 
>>> ===================================================
>>> Ioan Raicu
>>> Ph.D. Candidate
>>> ===================================================
>>> Distributed Systems Laboratory
>>> Computer Science Department
>>> University of Chicago
>>> 1100 E. 58th Street, Ryerson Hall
>>> Chicago, IL 60637
>>> ===================================================
>>> Email: iraicu at cs.uchicago.edu
>>> Web:   http://www.cs.uchicago.edu/~iraicu
>>> http://dev.globus.org/wiki/Incubator/Falkon
>>> http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
>>> ===================================================
>>> ===================================================
>>>
>>>   
>>
>