<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
This might be so, but when a user comes across behavior that is
significantly sub-optimal (such as sending few jobs that don't utilize
all the nodes at a site), they will want knobs to manually tune things
to be closer to optimal (in their opinion). That said, you are
probably right that the default setting should be completely automated,
but there should be knobs that can be turned on, off, up, down, etc...
to allow the user to avoid the bad behavior. For example, this means
allowing the user to turn off site scoring. <br>
<br>
This is not the first time we are having this discussion, and I only
brought up these points again since Ben started up the discussion. I
think we all have our opinions, and in the end, I am not the one who
will be implementing these knobs, so feel free to do what you think is
best!<br>
<br>
Ioan<br>
<br>
Mihael Hategan wrote:
<blockquote cite="mid:1193607722.25186.17.camel@blabla.mcs.anl.gov"
type="cite">
<pre wrap="">On Sun, 2007-10-28 at 15:05 -0500, Ioan Raicu wrote:
</pre>
<blockquote type="cite">
<pre wrap="">But my argument was, and still is, if there is only one site to submit
to, changing situations are almost irrelevant,
</pre>
</blockquote>
<pre wrap=""><!---->
Missed that. It is not irrelevant. The speed/capacity of a service is
determined by: the jobs you submit, the jobs others submit, the specific
type of hardware, and the load on the service node (and other things
like network latency). The jobs other submit and the load on the service
node vary with time. The bad thing about them is that it's hard to
predict how they affect things.
Furthermore, user specified rates suffer fundamentally from the problem
of the user having to understand how the whole thing works and picking
good values. What I've observed is that this doesn't work very well.
</pre>
<blockquote type="cite">
<pre wrap=""> as there are no options anyhow. Give me one example, where you have
only 1 site, set X and Y properly, yet you need site scores as an
additional throttling mechanism!
Mihael Hategan wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Sun, 2007-10-28 at 11:23 -0500, Ioan Raicu wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I mentioned 2 throttling mechanisms, one is to have X outstanding jobs
at any given time (limits jobs in the queue), and Y jobs/sec
submit rate (limits the rate of submission). I believe both of these
throttling mechanisms could exist without computing site scores,
assuming the user knows what to set X and Y to.
</pre>
</blockquote>
<pre wrap="">They do exist, but they don't deal with asymmetries between sites. Nor
do they deal with changing situations.
</pre>
<blockquote type="cite">
<pre wrap="">Ioan
Mihael Hategan wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Sun, 2007-10-28 at 10:25 -0500, Ioan Raicu wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Assuming you have a single site to submit to, then I don't see why you
don't want to disable the site scoring altogether?
</pre>
</blockquote>
<pre wrap="">Because having too many jobs on that one site may still cause problems.
That said, the algorithm currently there needs some work.
</pre>
<blockquote type="cite">
<pre wrap="">Of course you still want throttling, but that is more on the level
of X outstanding jobs at any given time (and possibly Y jobs/sec
submit rate), so you don't overrun the LRM, but you would not want to
lower X to some low value just because some jobs are failing. Again,
once you go to multi-site runs, you need the site scoring to decide
among the different sites, but with a single site, I see no drawbacks
to disabling the site scoring mechanism.
Ioan
Ben Clifford wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Sun, 28 Oct 2007, Ioan Raicu wrote:
</pre>
<blockquote type="cite">
<pre wrap="">they were due to the stale NFS handle error. I think Mihael outlined in an
email a while back how to disable the task submission throttling due to a bad
score, assuming that you have a single site to submit to anyways.
</pre>
</blockquote>
<pre wrap="">I know how to disable it. I don't particularly want it running rate free.
Whats happening here is that the feedback loop feeding back too much / too
fast for the situation I experience.
There's plenty of fun to be had experimenting there; and I suspect there
will be no One True Rate Controller.
</pre>
</blockquote>
<pre wrap="">--
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: <a class="moz-txt-link-abbreviated" href="mailto:iraicu@cs.uchicago.edu">iraicu@cs.uchicago.edu</a>
Web: <a class="moz-txt-link-freetext" href="http://www.cs.uchicago.edu/~iraicu">http://www.cs.uchicago.edu/~iraicu</a>
<a class="moz-txt-link-freetext" href="http://dsl.cs.uchicago.edu/">http://dsl.cs.uchicago.edu/</a>
============================================
============================================
_______________________________________________
Swift-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a>
<a class="moz-txt-link-freetext" href="http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel">http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">--
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: <a class="moz-txt-link-abbreviated" href="mailto:iraicu@cs.uchicago.edu">iraicu@cs.uchicago.edu</a>
Web: <a class="moz-txt-link-freetext" href="http://www.cs.uchicago.edu/~iraicu">http://www.cs.uchicago.edu/~iraicu</a>
<a class="moz-txt-link-freetext" href="http://dsl.cs.uchicago.edu/">http://dsl.cs.uchicago.edu/</a>
============================================
============================================
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">--
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: <a class="moz-txt-link-abbreviated" href="mailto:iraicu@cs.uchicago.edu">iraicu@cs.uchicago.edu</a>
Web: <a class="moz-txt-link-freetext" href="http://www.cs.uchicago.edu/~iraicu">http://www.cs.uchicago.edu/~iraicu</a>
<a class="moz-txt-link-freetext" href="http://dsl.cs.uchicago.edu/">http://dsl.cs.uchicago.edu/</a>
============================================
============================================
</pre>
</blockquote>
<pre wrap=""><!---->
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: <a class="moz-txt-link-abbreviated" href="mailto:iraicu@cs.uchicago.edu">iraicu@cs.uchicago.edu</a>
Web: <a class="moz-txt-link-freetext" href="http://www.cs.uchicago.edu/~iraicu">http://www.cs.uchicago.edu/~iraicu</a>
<a class="moz-txt-link-freetext" href="http://dsl.cs.uchicago.edu/">http://dsl.cs.uchicago.edu/</a>
============================================
============================================</pre>
</body>
</html>