[Swift-devel] Re: replication vs site score

[Mihael Hategan]

On Thu, 2009-04-09 at 10:32 -0700, Ioan Raicu wrote:

> > A better solution is to dynamically allocate workers in larger blocks if
> > you don't have arbitrary granularity on the allocation sizes. It
> > provides the middle ground that meets the scheduling system constraints
> > and minimizes inefficiencies.
> >   
> When you allocate 1 node at a time, in dynamic provisioning, its
> trivial to de-allocate nodes/workers, with a timeout. When you
> allocate N nodes in a single job, where N>1, de-allocation is not
> trivial anymore.

Exactly.

>  If a worker simply de-allocates (exit the process), that node remains
> allocated from the LRM's perspective, but de-allocated from the
> Coaster/Falkon perspective. When all N nodes are de-allocated, then
> the N nodes are released to the LRM. That is potentially a great deal
> of wastage. The better solution would be for there to be a centralized
> manager, that keeps track of an entire job (N nodes) and their
> utilization, and decide to de-allocate the entire N nodes at the same
> time, from both Coaster/Falkon and the LRM. Falkon doesn't support
> this unfortunately. Does Coaster support this?

That was the thing Mike was pushing for and which I started thinking of.
It's good that we're having this discussion, because it uncovers some of
the details involved.

>  If not, then I'd say that dynamic resource provisioning has to be
> kept at jobs of a single node level,

That does not follow. The mutually-exclusive choices you are presenting
are 1) no current support for block allocations, 2) single node
allocations and you're missing 3) future support for block allocations.
If (1) is false (2) is not the only alternative.

>  and not bunch together multiple node requests per job. This will
> obviously limit the use of dynamic provisioning for large scale runs,
> to LRMs that support large number of job submissions, proportional to
> the scale of the runs.