[Swift-devel] Re: provider staging stage-in rate on localhost and PADS

[Mihael Hategan]

So I'm running some tests.

So far here's how the stage-ins look like:
-site: local (my laptop)
-job input size: 32MB
-256 jobs
-job output size 0B
-measurement is made at the interface between service and worker (the
TCP connection). It is aggregated for all workers.
-2 workers, 4 jobs per worker
-this is the fast branch, but the job throughput is pretty irrelevant
here since this is I/O bound.

What I get is this:
file:
[IN]: Total transferred: 641.93 KB, current rate: 0 B/s, average rate:
4.62 KB/s
[OUT] Total transferred: 8 GB, current rate: 40.2 MB/s, average rate:
58.92 MB/s
Final status:  time:140732  Finished successfully:256
Time: 142.13, rate: 1 j/s

proxy:
Final status:  time:113915  Finished successfully:256
Time: 115.393, rate: 2 j/s
[IN]: Total transferred: 705.62 KB, current rate: 6.44 KB/s, average
rate: 6.24 KB/s
[OUT] Total transferred: 8 GB, current rate: 36.08 MB/s, average rate:
72.54 MB/s

(It is funny that proxy is faster than "file", but for now I'll ignore
that).

For comparison:
mike at blabla2 coasters$ time dd if=/dev/zero of=~/tmp/8g bs=32KB
count=262144
262144+0 records in
262144+0 records out
8388608000 bytes (8.4 GB) copied, 49.9836 s, 168 MB/s

Things could be improved, but staging in does not seem to be the
bottleneck. 
Next, stage-outs...

Mihael




On Wed, 2011-01-12 at 17:05 -0800, Mihael Hategan wrote:
> On Wed, 2011-01-12 at 15:57 -0600, Daniel S. Katz wrote:
> > As I read this, it appears that there is some limit in Swift, rather than in the hardware, that is causing these numbers to be very low.
> > 
> > Mihael, do you agree?
> 
> Yes, but that does not exclude a limit in hardware in other
> configurations.
> 
> >   Can you help us figure out what's going on?
> 
> Of course.
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel