[Swift-devel] 0.9 release notes draft

[Ben Clifford]

Below are the draft release notes for 0.9. They need more formatting, but 
all the text is there.

I have said little about coasters other than document two new 
configuration parameters and 'substantial ongoing development work'. If 
there are any specific additional points to be made there, comment here.

======


User interface
==============

*** Added console text interface to provide live information about swift
    runs, which can be enabled with the -tui commandline parameter

*** when replication is enabled, swift will locally kill jobs that have
    run for twice their specified walltime

Execution modes
===============

*** Support for Condor-G submissions, by setting a job specification attribute
    of "grid" and specifying a gridResource attribute containing the string
    to be placed into the Condor-G grid_resource classad.

*** Support for submissions to a local condor pool

*** Coasters: substantial ongoing development work

Configuration parameters
========================

*** Environment variable SWIFT_EXTRA_INFO, when set in an environment profile,
    is executed with the result being recorded in wrapper logs. This allows
    arbitrary information about the remote environment to be gathered and
    returned to the submit side.

*** New configuration option wrapper.invocation.mode, specifiable either
    globally in the configuration file or per-site as a profile key,
    that configures whether wrapper script invocations are made with an
    absolute path (as was the behaviour in Swift 0.8) or with a relative
    path (as was behaviour in previous versions of Swift).

***     coasterWorkerMaxwalltime - a coaster parameter to explicitly set worker maxwalltime, overriding the default computation based on job walltimes. this should be useful when it is desirable to specify worker parameters based on the known properties of the target queue rather than on properties of the jobs to be executed inside coasters.

***     coasterInternalIP parameter that allows the IP address used by coaster workers to communicate to the coaster head job to be set explicitly. this can be used when running on a cluster which an internal network which cannot access the IP address that the head node picks

*** TODO maxheapsize setting on commandline...

*** property ticker.disable to disable runtime ticker display

*** strsplit function which will split the input string based on separators that match the given pattern and return a string array.

New commands
============

*** The log-processing code, primarily exposed as the swift-plot-log command,
    has been merged into the main Swift release, rather than being a separate
    download.

*** There is a new command swift-osg-ress-site-catalog which will generate
    a site catalog based on data gathered from OSG's ReSS information
    system. This can be used by OSG users to easily generate a large number
    of site catalog entries for sites that are likely to work.

Language changes
================

*** Procedure invocations can be made in any expression context now, rather
    than only directly in an assignment.

*** Mappings can now be made in any declaration, whether it has an assignment
    or not. Previous a procedure call assignment and a mapping could not be
    made in the same statement.

*** Handling of [*] and . has changed. [*] has been broken for several
    Swift releases, but now has some approximation of its former behaviour,
    in a cleaner implementation.
    [*] is now an identity operation, that is array[*] == array.
    The structure access operator . can now take an array on the left
    hand side. In situations where a[*].img was used, it is permissible
    to continue to write a[*].img, or to write a.img - both of these will
    return an array of img elements.

*** Tighter type checking on app blocks. Primitive types and arrays of
    primitive types are permitted. Other types are prohibited.

*** Arrays of primitive types can be passed to app blocks, and will be
    expanded into multiple command-line parameters, one parameter per
    array element.

*** != operator had been broken forever. It works now.

*** output of trace() is changed for non-primitive datasets:

    * Array handling:
    Previous to this, trace would show the Java object representation of a (Future)PairIterator for arrays, which is fairly useless for a user. This commit makes trace show more about the array, and only emit the trace when the array is closed.

    * Other datasets:
    Trace will wait for those datasets to be closed, and emit their default
    string representation (including the variable name and path used in
    SwiftScript)

*** loop condition in iterate statements can now refer to variables declared within the iteration body (this was bug 177)

***    Mappings can now be made in any declaration, whether it has an assignment or not.



Deprecations and removal of old functionality
=============================================

*** Removed support for .dtm file extension which was deprecated in Swift 0.4

Internal changes
================

*** The wrapper.sh and seq.sh scripts that are deployed to remote sites to
    help with execution have been renamed to more Swift specific names, to
    avoid collision with user-supplied files of the same name. The new names
    are _swiftwrap and _swiftseq

*** Recompilation will happen if a .kml file was compiled with a different
    version of Swift to the version being invoked. This is in addition to the
    existing behaviour where a .swift file will be recompiled if it is newer
    than the corresponding .kml file.

*** Added a throttling parallelFor and changed the swift foreach underlying
    implementation to it. The throttling parallelFor limits the number of
    concurrent iterations allowed, which should allow swift to scale better
    in certain cases. This is controlled by the configuration property
    foreach.max.threads

--