[Swift-devel] Re: provider staging stage-in rate on localhost and PADS

[Mihael Hategan]

On Sun, 2011-01-16 at 18:56 -0600, Michael Wilde wrote:
> Mihael,
> 
> If I follow this right, your stageout rate (7MB/sec) is about what Allan saw for stage ins (~ 6MB/sec as I recall).

That is also close to what I get if I disable read-ahead for stage-ins.

> 
> But your stagein rate is 59MB/sec and 73MB/sec depending on the mode (file vs proxy).

That was experiment error. They both average around 60MB/s after
repeated runs. Though in the local/local case "file" has considerably
lower memory consumption that "proxy".

> 
> So that does not yet replicate what he's seeing (on the WAN).

No. Just local. I wanted to see if the code was introducing a
significant bottleneck, but it does not seem so. Though keep in mind
that this was on an SSD. An HDD may not get the same performance. I gave
the raw dd numbers for comparison. In the local/local case, one should
probably divide that by two since with swift you are both reading and
writing the data at the same time, while the dd test was from /dev/zero.

> 
> Allan, I cant recall if you sent numbers for local-host tests?
> 
> - Mike
> 
> 
> 
> 
> ----- Original Message -----
> > Right. So stageouts:
> > Progress: time:86039 Selecting site:231 Submitted:1 Active:8
> > Finished successfully:16
> > [IN]: Total transferred: 597.1 MB, current rate: 8.99 MB/s, average
> > rate: 7.02 MB/s
> > [OUT] Total transferred: 161.33 KB, current rate: 0 B/s, average rate:
> > 1.9 KB/s
> > 
> > That's probably because, as opposed to when the Java side reads files,
> > there is no read-ahead done by the worker. I'll see if I can add that.
> > 
> > On Sun, 2011-01-16 at 15:21 -0800, Mihael Hategan wrote:
> > > So I'm running some tests.
> > >
> > > So far here's how the stage-ins look like:
> > > -site: local (my laptop)
> > > -job input size: 32MB
> > > -256 jobs
> > > -job output size 0B
> > > -measurement is made at the interface between service and worker
> > > (the
> > > TCP connection). It is aggregated for all workers.
> > > -2 workers, 4 jobs per worker
> > > -this is the fast branch, but the job throughput is pretty
> > > irrelevant
> > > here since this is I/O bound.
> > >
> > > What I get is this:
> > > file:
> > > [IN]: Total transferred: 641.93 KB, current rate: 0 B/s, average
> > > rate:
> > > 4.62 KB/s
> > > [OUT] Total transferred: 8 GB, current rate: 40.2 MB/s, average
> > > rate:
> > > 58.92 MB/s
> > > Final status: time:140732 Finished successfully:256
> > > Time: 142.13, rate: 1 j/s
> > >
> > > proxy:
> > > Final status: time:113915 Finished successfully:256
> > > Time: 115.393, rate: 2 j/s
> > > [IN]: Total transferred: 705.62 KB, current rate: 6.44 KB/s, average
> > > rate: 6.24 KB/s
> > > [OUT] Total transferred: 8 GB, current rate: 36.08 MB/s, average
> > > rate:
> > > 72.54 MB/s
> > >
> > > (It is funny that proxy is faster than "file", but for now I'll
> > > ignore
> > > that).
> > >
> > > For comparison:
> > > mike at blabla2 coasters$ time dd if=/dev/zero of=~/tmp/8g bs=32KB
> > > count=262144
> > > 262144+0 records in
> > > 262144+0 records out
> > > 8388608000 bytes (8.4 GB) copied, 49.9836 s, 168 MB/s
> > >
> > > Things could be improved, but staging in does not seem to be the
> > > bottleneck.
> > > Next, stage-outs...
> > >
> > > Mihael
> > >
> > >
> > >
> > >
> > > On Wed, 2011-01-12 at 17:05 -0800, Mihael Hategan wrote:
> > > > On Wed, 2011-01-12 at 15:57 -0600, Daniel S. Katz wrote:
> > > > > As I read this, it appears that there is some limit in Swift,
> > > > > rather than in the hardware, that is causing these numbers to be
> > > > > very low.
> > > > >
> > > > > Mihael, do you agree?
> > > >
> > > > Yes, but that does not exclude a limit in hardware in other
> > > > configurations.
> > > >
> > > > >   Can you help us figure out what's going on?
> > > >
> > > > Of course.
> > > >
> > > >
> > > > _______________________________________________
> > > > Swift-devel mailing list
> > > > Swift-devel at ci.uchicago.edu
> > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > >
> > >
> > > _______________________________________________
> > > Swift-devel mailing list
> > > Swift-devel at ci.uchicago.edu
> > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > 
> > 
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>