[Swift-devel] How to increase Swift-coaster task rate

[Mihael Hategan]

Can I see the log?

I don't think that there is a set of things that I can easily point out,
but I can try to see what the problems might be.

What's the submit host (cpu/cores/mem, etc)?

Separate service or auto/local?

Mihael

On Thu, 2012-04-26 at 20:37 -0500, Michael Wilde wrote:
> Mihael,
> 
> David, Jon, and I are working on Cray benchmarks for a paper for the Cray Users Group.
> 
> In tests so far, we are being limited by job submission rates of about 80 tasks/sec.
> 
> We'd like very much to drive that up closer to 200/sec if at all possible for the benchmarks we're trying to run.
> 
> The current tests are doing sleep 0 jobs with no file transfer to about 2400 cores on a Cray benchmark system. The workdir is set to /dev/shm. The throttles are almost all set way up (Jon can post the specific config and values).
> 
> One thing we have not yet done is try to get the log traffic way down; thats next up to try.
> 
> We'll revert to testing against 480 cores on raven for now. That should still be enough to push the upper limit of Swift, Karajan and coasters.
> 
> Can you give us a set of things to check (set, turn off, etc) to try to get closer to 200 tasks/sec? Do we need to set <scratch> to /dev/shm in addition to work dir?
> 
> This latest run was I think with provider staging and pin coaster files.
> 
> Thanks,
> 
> - Mike
> 
>