[Swift-devel] Progress Report of Swift on BGP

[Zhao Zhang]

Hi, All

I made 2 small changes to wrapper.sh. For now, instead of creating 
everything on GPFS, wrapper.sh only create that id.success file on GPFS, 
leaving others to CN's ramdisk.

For improvements you could compare
http://www.ci.uchicago.edu/~zzhang/report-sleep-20080724-0030-3zbv20j6/
and
http://www.ci.uchicago.edu/~zzhang/report-sleep-20080724-1816-x10hf3s1/
Those two logs are both sleep_30x16384 on 8192 cores

I took stats from 1 pset as example:
OLD wrapper.sh

=== execute2 duration statistics for bgp0 ===
Total number of events: 501
Shortest event (s): 114.388999938965
Longest event (s): 356.953999996185
Total duration of all events (s): 118968.562002659
Mean event duration (s): 237.462199606105
Standard deviation of event duration (s): 53.4422013666663
Maximum number of events at one time: 311


NEW wrapper.sh

=== execute2 duration statistics for bgp000 ===
Total number of events: 519
Shortest event (s): 52.1200001239777
Longest event (s): 251.282999992371
Total duration of all events (s): 68348.8350019455
Mean event duration (s): 131.693323703171
Standard deviation of event duration (s): 39.0439215239727
Maximum number of events at one time: 321

Comparing the Mean event duration, we could see a 2X improvement.


Plus, I did a sleep_600 x 16384 test on 8192 cores with this new 
wrapper.sh. Things are perfect.
The log is at
http://www.ci.uchicago.edu/~zzhang/report-sleep-20080724-2357-4z97cn3f/

I also paste stats of 1 pset here

=== execute2 duration statistics for bgp000 ===
Total number of events: 518
Shortest event (s): 601.161000013351
Longest event (s): 1103.05599999428
Total duration of all events (s): 351741.216003895
Mean event duration (s): 679.037096532615
Standard deviation of event duration (s): 114.593631279296
Maximum number of events at one time: 293

The average event duration is 679 seconds, thus we have a 88% efficiency.
This is all my work about SWIFT on BGP today.

best wishes
zhangzhao