[Swift-devel] workflow run with the newest Swift

[Ben Clifford]

On Fri, 27 Jun 2008, lixi at uchicago.edu wrote:

> >Do you have the log file for this? I would like to see how 
> it worked.
> 
> It is on CI:
> /home/lixi/newswift/test/newversion/workflowtest-20080627-
> 1246-aba0l8d9.log

Some stats for that run:

Almost exactly 1h runtime.

Peak of 45 jobs actually running on workers at any one time; for most of 
the hour between 30 and 45 jobs were running on workers. At peak, around 
70 jobs were either queued or running.

The bulk of jobs took bang on 60s to execute on workers, with a relatively 
small number taking much less or more (up to 75s for some) - that is 
plotted in the 'info duration histograph' graph in the below URL.

13 different sites were tried. There is a breakdown of how much each was 
used in 'sites/success table' in the below URL. The decent sites got 200 
or so jobs each.

The log analyser doesn't deal with replication properly yet, but it looks 
like around 10% of jobs (about 200) got replicated.

Looking at per-site stats, it looks like most sites did not get more than 
6 jobs queued/running at once, rather than increasing. One site got 12 
though. I'm interested to know what caused that. I think it is your choice 
of jobThrottle parameter in your sites file generation. My rule of thumb 
has been 20 jobs at once for GRAM2 and a few hundred at once for GRAM4; 
are you specifying a lower job throttle in your submissions?

All the plots and stats are here:

http://www.ci.uchicago.edu/~benc/report-workflowtest-20080627-1246-aba0l8d9/

--