[Swift-devel] swift changing walltime of prews-gram jobs

[Allan Espinosa]

Or having different queue names for the coaster and workers. So for
example, I indicate "long" queue for the site coaster and "fast" for
the real jobs.

-Allan

On Sun, Jan 25, 2009 at 7:54 AM, Ben Clifford <benc at hawaga.org.uk> wrote:
>
> Using coasters will cause job submissions with different walltimes than
> your individual swift-level jobs.
>
> Coaster workers get submitted with a longer walltime than the jobs you are
> trying to send through. This is intended to result in coaster workers that
> will run long enough to run many jobs.
>
> At the moment, this is not very configurable. In the source code,
>
> provider-coaster//src/org/globus/cog/abstraction/coaster/service/job/manager/WorkerManager.java
>
> contains these fragments:
>
>    public static final Seconds TIME_RESERVE = new Seconds(60);
>    public static final int OVERALLOCATION_FACTOR = 10;
>    startWorker(new Seconds(req.maxWallTime.getSeconds())
>                            .multiply(OVERALLOCATION_FACTOR)
>                            .add(TIME_RESERVE), req.prototype);
>
>
> so whatever your maxwalltime is, you'll get coaster workers submitted with
> ten times that plus one minute.
>
> The coaster workers don't enforce job maxwalltimes, so you can work around
> this by making the job maxwalltimes small enough so that 10*that+60s fits
> inside the queue maximum wall time, even if that is actually too small for
> your jobs.
>
> You should see the same behaviour using local:pbs, which will use direct
> PBS submission instead of GRAM; but you don't. That is an inconsistency
> that suggests something is not right. My initial suspicion would be that
> the cog PBS provider is not correctly passing either the walltime or queue
> parameters. I will investigate this.
>
> Probably coasters should get another configuration option to allow the
> worker wall time to be more explicitly set, separately from job execution
> wall times - that makes sense for sites where parameters such as queue
> limits are well known by the user.


-- 
Allan M. Espinosa <http://allan.88-mph.net/blog>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>