<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
I think that all of these issues will go away soon, when we start using
the dynamic provisioning code that Ioan is working on. So I wonder if
they are worth worrying about too much?<br>
<br>
Ian.<br>
<br>
Mihael Hategan wrote:
<blockquote cite="mid1173313254.6939.5.camel@blabla.mcs.anl.gov"
type="cite">
<pre wrap="">So this limit would have to be a per-site limit.
There is no such thing right now. You can limit the total number of
concurrent jobs, but it's not exposed through swift.properties.
In libexec/scheduler.xml, you can try adding the following thing inside
<scheduler>...</scheduler>:
<property name="maxSimultaneousJobs" value="384"/>
Mihael
On Wed, 2007-03-07 at 17:27 -0600, Veronika V. Nefedova wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Right. Teragrid at NCSA has the limit of 384 queued or running jobs per user.
Nika
At 05:19 PM 3/7/2007, Mihael Hategan wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Wed, 2007-03-07 at 16:58 -0600, Veronika V. Nefedova wrote:
</pre>
<blockquote type="cite">
<pre wrap="">OK, Here is my another question.
Teragrid allows the user to have 385 jobs in a queue. If I run my complete
workflow (244 molecules), on stage four I'll have 80 times 244 jobs (i.e.
close to 20K). How do I set the limit for the number of submitted jobs to
the queue to 385 ? I remember that condor had a specific parameter to
condor_submit that was managing exactly that...
</pre>
</blockquote>
<pre wrap="">Is this 385 jobs per site?
</pre>
<blockquote type="cite">
<pre wrap="">Nika
At 04:36 PM 3/7/2007, Mihael Hategan wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Wed, 2007-03-07 at 16:30 -0600, Veronika V. Nefedova wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
I've noticed one very strange behavior. For example, I have 68 jobs
</pre>
</blockquote>
</blockquote>
</blockquote>
<pre wrap="">to be
</pre>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">submitted to the remote host simultaneously. Swift submits at first
</pre>
</blockquote>
<pre wrap="">just 26
</pre>
<blockquote type="cite">
<pre wrap="">jobs. I checked that several times - its always 26 jobs. Then, when at
least one job out of those 26 is finished - swift goes ahead and
</pre>
</blockquote>
</blockquote>
</blockquote>
<pre wrap="">submits
</pre>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">the rest (all of those left - 42 in my case).
Is it a bug or a feature?
</pre>
</blockquote>
<pre wrap="">Feature. Although it should probably be tamed down in the one site case.
Each site has a score that changes based on how it behaves. If a site
completes jobs ok, it gets a higher score in time. If jobs fail on it,
it gets a lower score.
Now, let's consider the following scenario: 2 sites, one fast one slow.
With no scores and no limitations, half of the jobs would go to one, and
half to the other. The workflow finishes when the slow site finishes
half the jobs.
What happens however, is that Swift limits the number of initial jobs,
and does "probing". This allows it to infer some stuff about the sites
by the time it gets to submit lots of jobs. It should yield better
performance on larger workflows with imbalanced sites, which is, I'm
guessing, our main scenario.
</pre>
<blockquote type="cite">
<pre wrap="">Nika
_______________________________________________
Swift-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a>
<a class="moz-txt-link-freetext" href="http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel">http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel</a>
</pre>
</blockquote>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap=""><!---->
_______________________________________________
Swift-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a>
<a class="moz-txt-link-freetext" href="http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel">http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Ian Foster, Director, Computation Institute
Argonne National Laboratory & University of Chicago
Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439
Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637
Tel: +1 630 252 4619. Web: <a class="moz-txt-link-abbreviated" href="http://www.ci.uchicago.edu">www.ci.uchicago.edu</a>.
Globus Alliance: <a class="moz-txt-link-abbreviated" href="http://www.globus.org">www.globus.org</a>.
</pre>
</body>
</html>