[Swift-devel] Re: replication vs site score

[Ioan Raicu]

Aha, but I think the predictions are upper bounds, not upper and lower 
bounds. In essence, when they predict that your job will wait for 11.2 
hours, with 95% confidence, and your job runs in 15 minutes, then in no 
way have they made a prediction in error. Now, if they would have 
predicted 1 minute, and it took 15 minutes, then it would have been an 
error. It is possible that they do not use knowledge of back-filling, 
which would make small jobs run immediately, although they would predict 
a long queue wait time, as if no back-filling is enabled. Its not clear 
how customized the predictor is, to the scheduler and features of the 
LRM, so there is certainly room for being pessimistic on their predictions.

Ioan

Mihael Hategan wrote:
> On Wed, 2009-04-08 at 13:58 -0700, Ioan Raicu wrote:
>   
>> Mihael Hategan wrote: 
>>     
>>> You're right. I was trying to say that fundamentally the problem of
>>> uncertainty in queue times will remain by virtue of the fact that the
>>> times when people submit jobs (as well as the amount of jobs) is
>>> unpredictable and it can affect other people's job queue times. 
>>>
>>> The predictor in the paper answers the question "if you were to submit
>>> your job before the state of the queue changes in any way, what would be
>>> the expected queue time for the job" and not "what will be the queue
>>> time for the job".
>>>
>>>   
>>>       
>> Yes, its possible that between a query of prediction, and actual
>> submission, the state of the queues change, and therefore the actual
>> result change. But, every prediction comes with some error bounds, so
>> its possible that the change in queue state, might be reflected in the
>> error bars.
>>     
>
> I don't know... The system predicted that a 2 minute job on Abe would
> sit 11.2 hours in the queue and 2.4 hours on QueenBee, but I've ran 20
> such jobs on both in the past 15 minutes.
>
>
>
>   

-- 
===================================================
Ioan Raicu, Ph.D.
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
http://dev.globus.org/wiki/Incubator/Falkon
http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
===================================================
===================================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20090408/6cfb5f97/attachment.html>