[Swift-devel] Re: replication vs site score

[Mihael Hategan]

On Thu, 2009-04-09 at 10:04 -0700, Ioan Raicu wrote:
> There are use cases where static resource allocation are better than
> dynamic ones. Again, we come back to the BG/P system. There is a
> policy that only allows you to submit X number of jobs to Cobalt, and
> X is < 10 jobs. Now, if you want to allocate resources dynamically, in
> smaller chunks, you are limited to only a few jobs. Static
> provisioning all of a sudden seems attractive.

It's a valid scenario and a valid solution, but asserting that it's the
only solution or that it's the best solution seems inappropriate.

A better solution is to dynamically allocate workers in larger blocks if
you don't have arbitrary granularity on the allocation sizes. It
provides the middle ground that meets the scheduling system constraints
and minimizes inefficiencies.

> 
> Another thing that you have to remember, that for some systems, like
> the BG/P, getting 2 allocations of 64 nodes each, is not the same as
> getting 1 allocation of 128 nodes. The 1 single allocation of 128
> nodes has networking configured in such a way to allow node-to-node
> communication efficiently. The 2 separate allocations, could be
> allocated in completely opposite ends of the system, and hence having
> poor networking properties to do node-to-node communication, between
> the separate allocations (if its even possible, I am not sure, the
> networks might be completely separate). This might not be important
> for vanilla Swift, but some of the MTDM work (previously known as
> collective I/O) relies on good network connectivity between any node
> in the allocation to pass data around and avoiding GPFS.

I'm not sure what dynamic vs. static allocation of workers, in
principle, has to do with the implementation hurdles of CIO on the BG/P.

Different systems have different constraints. Dynamic allocation can be
made to adapt to those constraints.