[Swift-devel] Adjust site scores on job start not job end

[Michael Wilde]

We discussed in the Swift internals review meetings the desirability of adjusting the scheduler's site scores more by how many job start events than by sucessful completion events.

The rationale was that for workloads consisting entirely of long running jobs, on for example OSG, this approach would much more quickly reward sites that have been starting jobs with additional jobs, until the start rate diminishes when the jobs start queuing up.

Another approach we discussed (which was demonstrated by Dinah Sulakhe to be successful in VDS) was to keep sending jobs to sites until each site has some fixed threshold of jobs sitting in its queue, and to keep all the sites at some threshold (possibly a per-site threshold based on the site's throughput).

We're now at the point where a few users (Glen and Allan) would benefit from this change in scheduling algorithm.

Mihael, all, can you where and how to explore such changes (module-wise) and what pitfalls are likely to be encountered?

Thanks,

- Mike
 
----- Forwarded Message -----
From: "Michael Wilde" <wilde at mcs.anl.gov>
To: "Michael Wilde" <wilde at mcs.anl.gov>
Sent: Wednesday, August 25, 2010 2:22:16 PM GMT -06:00 US/Canada Central
Subject: Sched on start not end

-- 
Sent from my mobile device

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory