[Swift-devel] coaster hello world x 1000 and related.

[Ben Clifford]

These are some notes on some playing round with coasters and related stuff 
today - there's not any particular strong point made, but here are details 
for anyone interested.

I tried a 1000 x hello-world run. All of the below were submitted from my 
laptop on residential DSL in the UK. Most of the below were only measured 
once.

Running through GRAM4 to TG NCSA Mercury, a run takes about 8000 seconds 
with a jobThrottle of 4.

Running with coasters+GRAM2 with a job throttle of 0, to TG NCSA Mercury, 
this takes about 1500 seconds.

Running coasters with direct PBS submission of workers rather than using 
gram2 to submit the workers leads to various breakages - perhaps a problem 
with the pbs provider. I'll have to investigate that some more.

Then I noticed I was using the gt2 gridftp server for that site (which is 
GridFTP Server 2.5 (gcc64, 1182369948-63); I switched to the main one 
which is GridFTP Server 2.5 (gcc64, 1182369948-63) to see how that would 
change things. It doesn't seem to change much in this case.

I also increased the throttle up to 0.2, which is pretty much the maximum 
that GRAM2 is OK with - the runtime is down to less than 600s - something 
like 1/13th of the time taken to run in untuned GRAM4 mode. (Note that I 
expect fiddling with various throttles with GRAM4 would produce some 
decent speedup)

A lot of that 600s is taken up ramping up to threshold. I set the 
initialScore to 100 so that submission would start at a higher rate rather 
than ramping up. That didn't have much effect.

So at this point, most of the time seems to be taken up with file 
operations rather than job submission operations.

Next I doubled the file transfer and file operation throttles (to 8 and 16 
respectively). This lowers the run time after execution a bit, to around 
300s; however the realtime of the run went up to 600s, apparently because 
someone else woke up and started putting stuff in the queue on Mercury.

--