[Swift-devel] Re: provider staging stage-in rate on localhost and PADS

[Daniel S. Katz]

As I read this, it appears that there is some limit in Swift, rather than in the hardware, that is causing these numbers to be very low.

Mihael, do you agree?  Can you help us figure out what's going on?

Dan


On Jan 12, 2011, at 3:37 PM, Allan Espinosa wrote:

> setup1:
> 
> N workers on localhost (10, 20, 40)
> coaster service on localhost
> swift client on localhost
> 
> 10,000 2.3MB files in Localdisk (/var/tmp/RuptureVariations in communicado)
> jobs are simple 'cat inputfile' jobs.  no data is explicitly staged
> out in the app() invocation
> 
> Result:
> max transfer rate is 7MB/s with 10 workers resulting to a 2.5 jobs
> staged per second rate.
> 
> I attached the plot of bandwidth (red = 10 workers, green = 20
> workers, blue = 40 workers).
> 
> We maybe CPU bound at this point.  i'll try testing on
> communicado->bridled next.
> 
> setup2: (communicado->PADS)
> 
> N workers on PADS workers (10, 40)
> coaster service on communicadp
> swift client on communicado
> 
> pads.png shows the plot of bandwidth (max is around 5.5MB/s).
> legend: black = 10 workers, red = 40 workers
> 
> this translates to a jobrate of  2 jobs/ second
> 
> -Allan
> 
> -- 
> Allan M. Espinosa <http://amespinosa.wordpress.com>
> PhD student, Computer Science
> University of Chicago <http://people.cs.uchicago.edu/~aespinosa>
> <local.png><pads.png>

-- 
Daniel S. Katz
University of Chicago
(773) 834-7186 (voice)
(773) 834-3700 (fax)
d.katz at ieee.org or dsk at ci.uchicago.edu
http://www.ci.uchicago.edu/~dsk/