[Swift-devel] Coaster capabilities for release 0.9

[Mihael Hategan]

On Tue, 2009-04-21 at 14:12 -0500, Michael Wilde wrote:
> > Which should I rather do now, continue implementing block allocations,
>  > or sort out discussions about coasters from the mailing list?
> 
> You should do this:
> 
> First send a sentence or two on how block allocations are going: whats 
> involved, and how long you think they will take till checked in. What 
> kind of testing will be required to ensure functional and stable.

I haven't explicitly done so because I thought I mentioned this before.

I have an algorithm that I tested with various loads and configurations.
It can work with limited numbers of jobs, and pre-set allocations
(though there are some tweaks there that are still needed - a major one
being when to choose pre-existing allocations instead of automated
allocations; for example, if the pre-existing allocation is tomorrow,
and I submit now, I'd presumably want some jobs to be started before
tomorrow).

I need to now plug that into the rest of the coaster code (this is the
part I'm doing right now), find ways to specify parameters in the sites
file, efficiently ship those parameters remotely, and do some basic
testing. I estimate that to take somewhere in the range of one to two
weeks of development time, but that's not with very high confidence.

The kind of testing needed to ensure stability is not different from
what we've discussed and what we already know: we'll need to run
synthetic tests on various sites, and we'll need to run existing
applications on various sites. This is not new.

> 
> Then continue developing them.
> 
> Between now and Friday, do the list I asked for.
> 
>  > (i.e. would you please let me do my job?)
> 
> Yes.
> 
> I know development takes concentration. To achieve that, you manage your 
> time, and the rate at which you read and answer email or take any other 
> interrupts.
> 
> If something needs immediate attention, I or others say so, or your best 
> judgment does. If not, it doesn't, but needs eventual attention.
> 
> A list of coaster issues was started on swift-devel Feb 13:
> http://mail.ci.uchicago.edu/pipermail/swift-devel/2009-February/004511.html

There are 4 issues there:
- bootstrap issues (these are solved - this was discussed on the mailing
list)
- sites.xml attribute for java - the priority of this has become lower
because the detection is done better now, but it should probably still
be implemented (this was also on the mailing list as far as I can
remember; also, I think one can specify JAVA_HOME or make sure java is
in the path using the environment or otherwise)
- service on the worker node - after block allocations, unless Ben does
otherwise
- the scalability problem - we're doing block allocations mainly to
address this

> 
> A report on progress and summary of open design issues and work 
> remaining would benefit everyone.
> 
> Find a good stopping point in the next few days, then make list of work 
> items and design and testing issues on coasters.
> 
> Then discuss and get feedback. Then develop what you listed. Then 
> repeat. That *is* your job.
> 

I wish :)

Yep. We discussed, got feedback, and I was now developing. Discussing
the same issues again was interfering with my developing. So let me
re-state this: "I am now developing coaster block allocations. I will
send feedback from time to time, when I have interesting things to send
feedback about. I will probably have testable code in 1 to 2 weeks."