[Swift-devel] pilot job paper

[Michael Wilde]

Ian, QBETS sounds very useful; I met the developer at last year's SDCI meeting, and thought we could readily use QBETS predictions in Swift scheduling. Mihael, what would be a general interface from picking up site queue predictions from such tools with some kind of loose coupling? 


Also, in the Condor-G-based coaster factory approach we tested in ExTENCI, we could plug QBETS info into its factory script to bias the volume of pilot jobs we are sending to each site. Then the mod Mihael made recently in coaster scheduling will automatically keep all running coaster slots filled with active jobs. 



- Mike 

----- Original Message -----



E.g., I used it here: http://ianfoster.typepad.com/blog/2009/08/whats-fastera-supercomputer-or-ec2.html 




On Jul 21, 2011, at 9:18 AM, Ian Foster wrote: 



I think that you may find it useful to look at the queue time predictor that Rich Wolski developed. QBETS I think is the name? 


Ian. 



On Jul 20, 2011, at 4:41 PM, Ketan Maheshwari wrote: 









I need queuing time vs. advertised job walltime on various clusters 
(with various/random degrees of utilization). That's to see whether it's 
useful to have coasters at all. 

The number of jobs is an orthogonal dimension (i.e. we may want to 
measure the queuing time vs. #of jobs for various walltimes, but later). 
The actual job duration is not relevant. The amount of data is not 
relevant. 

Clouds are an interesting environment, but not for this particular 
problem. That's because we need to see how much it takes to acquire 
resources, not how fast some job middleware is after we got hold of 
those resources. 


Do you have any specific environments in mind for these experiments? For the requirement of various/random degrees of utilization, we can use MCS local cluster (10 x 64bit + 3 x 32 bit machines), Beagle, and Ranger. 

Regards, 
-- 
Ketan 


_______________________________________________ 
Swift-devel mailing list 
Swift-devel at ci.uchicago.edu 
https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel 

_______________________________________________ 
Swift-devel mailing list 
Swift-devel at ci.uchicago.edu 
https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel 


_______________________________________________ 
Swift-devel mailing list 
Swift-devel at ci.uchicago.edu 
https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel 



-- 
Michael Wilde 
Computation Institute, University of Chicago 
Mathematics and Computer Science Division 
Argonne National Laboratory 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110721/c5d63d3f/attachment.html>