[Swift-devel] pilot job paper

[Ian Foster]

I made the suggestion in the context of the discussion of data sources about historical queue times for a paper. But I can imagine it could be useful for scheduling too.


On Jul 21, 2011, at 11:12 AM, Michael Wilde wrote:

> Ian, QBETS sounds very useful; I met the developer at last year's SDCI meeting, and thought we could readily use QBETS predictions in Swift scheduling. Mihael, what would be a general interface from picking up site queue predictions from such tools with some kind of loose coupling?
> 
> Also, in the Condor-G-based coaster factory approach we tested in ExTENCI, we could plug QBETS info into its factory script to bias the volume of pilot jobs we are sending to each site. Then the mod Mihael made recently in coaster scheduling will automatically keep all running coaster slots filled with active jobs.
> 
> - Mike
> 
> E.g., I used it here: http://ianfoster.typepad.com/blog/2009/08/whats-fastera-supercomputer-or-ec2.html
> 
> 
> On Jul 21, 2011, at 9:18 AM, Ian Foster wrote:
> 
> I think that you may find it useful to look at the queue time predictor that Rich Wolski developed. QBETS I think is the name?
> 
> Ian.
> 
> On Jul 20, 2011, at 4:41 PM, Ketan Maheshwari wrote:
> 
> 
> 
> I need queuing time vs. advertised job walltime on various clusters
> (with various/random degrees of utilization). That's to see whether it's
> useful to have coasters at all.
> 
> The number of jobs is an orthogonal dimension (i.e. we may want to
> measure the queuing time vs. #of jobs for various walltimes, but later).
> The actual job duration is not relevant. The amount of data is not
> relevant.
> 
> Clouds are an interesting environment, but not for this particular
> problem. That's because we need to see how much it takes to acquire
> resources, not how fast some job middleware is after we got hold of
> those resources.
> 
> 
> Do you have any specific environments in mind for these experiments? For the requirement of various/random degrees of utilization, we can use MCS local cluster (10 x 64bit + 3 x 32 bit machines), Beagle, and Ranger. 
> 
> Regards,
> -- 
> Ketan
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> 
> 
> -- 
> Michael Wilde
> Computation Institute, University of Chicago
> Mathematics and Computer Science Division
> Argonne National Laboratory
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110721/09b4f9d8/attachment.html>