[Swift-devel] provider staging stage-in rate on localhost and PADS

[Mihael Hategan]

And then try to run this with very large files.

Essentially that will tell whether the limit is in the coaster i/o
bandwidth or the job throughput.

On Wed, 2011-01-12 at 17:06 -0600, Justin M Wozniak wrote:
> I'm actually trying to isolate this as well- can you try running this with 
> very small files?
> 
> On Wed, 12 Jan 2011, Allan Espinosa wrote:
> 
> > setup1:
> >
> > N workers on localhost (10, 20, 40)
> > coaster service on localhost
> > swift client on localhost
> >
> > 10,000 2.3MB files in Localdisk (/var/tmp/RuptureVariations in communicado)
> > jobs are simple 'cat inputfile' jobs.  no data is explicitly staged
> > out in the app() invocation
> >
> > Result:
> > max transfer rate is 7MB/s with 10 workers resulting to a 2.5 jobs
> > staged per second rate.
> >
> > I attached the plot of bandwidth (red = 10 workers, green = 20
> > workers, blue = 40 workers).
> >
> > We maybe CPU bound at this point.  i'll try testing on
> > communicado->bridled next.
> >
> > setup2: (communicado->PADS)
> >
> > N workers on PADS workers (10, 40)
> > coaster service on communicadp
> > swift client on communicado
> >
> > pads.png shows the plot of bandwidth (max is around 5.5MB/s).
> > legend: black = 10 workers, red = 40 workers
> >
> > this translates to a jobrate of  2 jobs/ second
> >
> > -Allan
> >
> >
>