[Swift-devel] Planning next set of Swift features and releases

[Michael Wilde]

Before 0.9 is released, I want to discuss an agreed upon sequence for 
the next set of enhancements and improvements, and a target set of dates 
and release numbers for them.

It is a minor issue, but I see Swift 1.0 as having some greater 
significance as a release number. You may or may not agree - but it 
should be discussed and we should make a conscious decision on whats 
next. Also minor but going from 0.9 to 0.10 is odd. 0.09 to 0.10 would 
have been OK. 0.91 .92 makes more sense, or 0.9.1. I dont much care at 
the moment.

The significance of calling something 1.0 thought will extend to funding 
issues, as our next NSF report will be due soon - June 1 I think - and 
concurrently I need to go to NSF and other agencies for continued 
funding, probably with an unsolicited proposal.

My preference is to do 2 or more point releases in the next 16 weeks and 
shoot for 1.0 by the end of August. The significance of that release is 
that it will be the last under the current round of NSF funding. So 
whether we call it 1.0, 1.1, whatever, it will be a milestone of sorts.

Below is a *very* rough draft of my features priorities. I want to ask 
others to make a list of features important to you, so we can sort and 
prioritize, and turn into a set of planned point releases.

Ian challenged me to narrow it to 10 features; I came close but have not 
yet given it the thought it needs, looked through bugzilla, etc. Then I 
backslid, but to pick 10, we need to consider more.

Serious effort on the feature and release list can start next week after 
0.9 (and after my current grant pressure eases up a bit).

- Mike

*** rough features on Mike's mind:

(These are not sorted into categories yet, not of similar size; A few 
more distant features crept in; others are missing)

Some of these require language deliberation, which makes them harder.

Some would make huge differences in usability (like better runtime 
errors, with line numbers. Also hard, but very valuable).

Clean up mapper model.

  - Names and interfaces (and maybe model):
  - pos params to make externs more compact;
  - single_file_mapper to take expressions in its short form.
  - re-consider mixed type mappings (file and scalar)

Structure assignment and related issues

   - remove limitations on this
   - maybe some similar language limits to address

Good run time error messages with source code line numbers

   - summary of errors at end of output is ineffective
   - review of messages in log that should go to output
   - review of message content in log

Coaster polishing and testing/robustness

   - address scheduler anomalies
   - 3 modes of provisioning

Condor-G support

tc.data and PATH extensions

  - rename tc.data (e.g.: applist.txt)?
  -  clean up its format, make simpler to add apps.

app() syntax review

  - trailing ";" confusing
  - quoting review and improvements if needed
  - multi-line

Auto app install

Make sites & tc config easy/instant for osg, tg, local clusters

  - productive ADEM

namespace::name:version for function/app identifiers

Global variables

Swift command

  - multi-site selection
  - run management (maybe a separate command)

Scaling

  - Efficient handling of 1M and more jobs per wf
  - what else needed here?

Library / #include feature

Provenance

  - make it end-user useful and ready
  - easy to get perf numbers
  - easy to associate swift parameters with files and runs

More logging / status reporting enhancement

  - enhance format and info content of log tools report

Review of iteration and related flow of control issues
  - why iterate seems hard to use
  - for, do, while?
  - functional equivalents


Built-in and/or intrinsic functions

  - remove @
  - additional string functions, perhaps other related builtins
  - (time, date, sizeof, ...)
  - Built-in function extensibility
  - general external function mechanism - shell out? lib path?

-- Longer term:

Coasters on bgp, kraken and sico

IO and Data management improvements

  - mtdm (cio) data management fully integrated, with extensibility
  - broadcast
  - pull
  - batched output
  - local data transfer bypass