[Swift-devel] submitting jobs to the queue

[Tiberiu Stef-Praun]

Even better:
Can we have a knob that could be used by the user to increase the
aggressiveness of the job submission rate ? Some users might enjoy
that (being more in control).

Tibi

On 3/9/07, Veronika  V. Nefedova <nefedova at mcs.anl.gov> wrote:
> Hi, Mihael:
>
> Is it possible to remove this feature in the one site case ? For example,
> the queue is now almost empty on TG, but I have to wait for 1.5 hours for
> the rest of my jobs to be submitted (thats the average running time of my
> job) - and the queue might be full by that time...
>
> Nika
>
> At 04:36 PM 3/7/2007, Mihael Hategan wrote:
> >On Wed, 2007-03-07 at 16:30 -0600, Veronika V. Nefedova wrote:
> > > Hi,
> > >
> > > I've noticed one very strange behavior. For example, I have 68 jobs to be
> > > submitted to the remote host simultaneously. Swift submits at first
> > just 26
> > > jobs. I checked that several times - its always 26 jobs. Then, when at
> > > least one job out of those 26 is finished - swift goes ahead and submits
> > > the rest (all of those left - 42 in my case).
> > > Is it a bug or a feature?
> >
> >Feature. Although it should probably be tamed down in the one site case.
> >Each site has a score that changes based on how it behaves. If a site
> >completes jobs ok, it gets a higher score in time. If jobs fail on it,
> >it gets a lower score.
> >
> >Now, let's consider the following scenario: 2 sites, one fast one slow.
> >With no scores and no limitations, half of the jobs would go to one, and
> >half to the other. The workflow finishes when the slow site finishes
> >half the jobs.
> >What happens however, is that Swift limits the number of initial jobs,
> >and does "probing". This allows it to infer some stuff about the sites
> >by the time it gets to submit lots of jobs. It should yield better
> >performance on larger workflows with imbalanced sites, which is, I'm
> >guessing, our main scenario.
> >
> > >
> > > Nika
> > >
> > > _______________________________________________
> > > Swift-devel mailing list
> > > Swift-devel at ci.uchicago.edu
> > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > >
>
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>


-- 
Tiberiu (Tibi) Stef-Praun, PhD
Research Staff, Computation Institute
5640 S. Ellis Ave, #405
University of Chicago
http://www-unix.mcs.anl.gov/~tiberius/