[Swift-commit] r7115 - tags/swift-0.94.1
davidk at ci.uchicago.edu
davidk at ci.uchicago.edu
Mon Sep 30 12:15:50 CDT 2013
Author: davidk
Date: 2013-09-30 12:15:50 -0500 (Mon, 30 Sep 2013)
New Revision: 7115
Added:
tags/swift-0.94.1/RELEASENOTES.txt
Removed:
tags/swift-0.94.1/CHANGES.txt
Log:
Release notes
Deleted: tags/swift-0.94.1/CHANGES.txt
===================================================================
--- tags/swift-0.94.1/CHANGES.txt 2013-09-30 03:20:35 UTC (rev 7114)
+++ tags/swift-0.94.1/CHANGES.txt 2013-09-30 17:15:50 UTC (rev 7115)
@@ -1,732 +0,0 @@
-(07/05/11)
-
-*** Provenance tracking in the compiled file is now disabled by default.
- Enable with swift -enable.provenance <script>
-
-*** Variables that go out of scope are now automatically cleaned. This
- mostly consists of nullifying various fields, but for the concurrent
- mapper it also means removal of files
-
-*** Fixed iterate so that iterate i {} until (i > 3); does not include
- an iteration in which i == 3.
-
-(07/01/11)
-
-*** Uninitialized variables are detected at compile time instead of causing
- a run to hang
-
-*** A previous commit to deal with future faults occurring in the middle of
- a deep copy would incorrectly handle multi-dimensional arrays. This is
- now fixed.
-
-(02/17/10)
-
-*** Fixed deadlock when iterating on an array that is being written to
- in a foreach loop
-
-(07/03/09)
-*** New import keyword which allows other SwiftScript source files to be
- imported.
-
-(07/01/09)
-*** New writeData procedure which writes data structures to files in the same
- format as used by readData.
-
-(06/30/09)
-*** New parameter -condor-g for swift-osg-ress-site-catalog which causes
- sites files to be generate which will use a local condor-g installation
- instead of making direct GRAM submissions.
-
-(06/25/09)
-*** 'global' keyword to allow top level declarations to be visible in all
- scopes.
-
-(05/17/09)
-*** Allow wrapper parameters to be specific in a file rather than on the
- commandline. This trades commandline length for an extra file to stage-in
- per job.
- A new parameter, wrapper.parameter.mode, is introduced to control this.
- This was feature request bug 201.
-
-(05/07/09)
-*** swift-log-to-cedps is removed. The functionality of generating
- .cedps format files is available by specifying the desired .cedps files
- in an invocation of swift-plot-log. Lack of widespread use of this
- command means a deprecation period is probably unnecessary.
-
-(05/05/09)
-*** New configuration property provenance.log which controls whether verbose
- provenance-related log information is recorded in the run log file or
- not. By default, this is false, which means that less information is
- output by default now than previously.
-
-(04/27/09)
-*** Swift 0.9 built from Swift SVN r2860 and cog SVN r2388
-
-(03/30/09)
-*** Removed support for .dtm file extension which was deprecated in Swift 0.4
-
-(03/29/09)
-*** Added console text interface to provide live information about swift
- runs.
-
-(03/27/09)
-*** Environment variable SWIFT_EXTRA_INFO, when set in an environment profile,
- is executed with the result being recorded in wrapper logs. This allows
- arbitrary information about the remote environment to be gathered and
- returned to the submit side.
-
-*** The wrapper.sh and seq.sh scripts that are deployed to remote sites to
- help with execution have been renamed to more Swift specific names, to
- avoid collision with user-supplied files of the same name. The new names
- are _swiftwrap and _swiftseq
-
-(03/15/09)
-*** The log-processing code, primarily exposed as the swift-plot-log command,
- has been merged into the main Swift release, rather than being a separate
- download.
-
-(03/09/09)
-*** Added a throttling parallelFor and changed the swift foreach underlying
- implementation to it. The throttling parallelFor limits the number of
- concurrent iterations allowed, which should allow swift to scale better
- in certain cases.
-
-(02/24/09)
-*** New configuration option wrapper.invocation.mode, specifiable either
- globally in the configuration file or per-site as a profile key,
- that configures whether wrapper script invocations are made with an
- absolute path (as was the behaviour in Swift 0.8) or with a relative
- path (as was behaviour in previous versions of Swift).
-
-(02/22/09)
-*** Recompilation will happen if a .kml file was compiled with a different
- version of Swift to the version being invoked. This is in addition to the
- existing behaviour where a .swift file will be recompiled if it is newer
- than the corresponding .kml file.
-
-(02/17/09)
-*** There is a new command swift-osg-ress-site-catalog which will generate
- a site catalog based on data gathered from OSG's ReSS information
- system. This can be used by OSG users to easily generate a large number
- of site catalog entries for sites that are likely to work.
-
-(02/14/09)
-*** Handling of [*] and . has changed. [*] has been broken for several
- Swift releases, but now has some approximation of its former behaviour,
- in a cleaner implementation.
- [*] is now an identity operation, that is array[*] == array.
- The structure access operator . can now take an array on the left
- hand side. In situations where a[*].img was used, it is permissible
- to continue to write a[*].img, or to write a.img - both of these will
- return an array of img elements.
-
-(02/09/09)
-*** Procedure invocations can be made in any expression context now, rather
- than only directly in an assignment.
-
-*** Mappings can now be made in any declaration, whether it has an assignment
- or not. Previous a procedure call assignment and a mapping could not be
- made in the same statement.
-
-(02/05/09)
-*** Tighter type checking on app blocks. Primitive types and arrays of
- primitive types are permitted. Other types are prohibited.
-
-*** Arrays of primitive types can be passed to app blocks, and will be
- expanded into multiple command-line parameters, one parameter per
- array element.
-
-(01/23/09)
-*** Swift 0.8 built from Swift SVN r2448 and cog SVN r2261
-
-(01/14/09)
-*** Application success/failure status reporting can now be done using
- CoG provider status, rather than the previous only choice of
- using status files on the shared file system. A status.mode parameter
- has been added to set this. It can be configured either in the
- swift.properties file, to have effect for all sites, or can be set
- per-site.
-
-(01/10/09)
-*** Console output for individual application invocation start and finish
- is no longer shown. The progress ticker now appears more often.
- This should give a better overview of run progress.
-
-*** More parallelisation of execution, so that some constructs which should
- have always work now actually work. For example, mappers can take
- values as parameters which are not known "early" in a run, and will
- have their initialisation deferred until those parameters are ready.
- SwiftScript programs which previously worked should still work; and
- some programs which did not previously work should now work.
-
-(11/11/08)
-*** Swift 0.7 built from Swift SVN r2318 and cog SVN r2255
-
-(08/15/08)
-*** Swift 0.6 built from Swift SVN r2177 and cog SVN r2127
-
-(06/06/08)
-*** SWIFT_JOBDIR_PATH environment variable can be use in profiles (or
- anywhere else that will get it to the worker script) to cause the
- worker to copy input files to a possibly-worker-node-local working
- directory.
-
-(04/24/08)
-*** PATHPREFIX environment variable is prefixed to the path before execution
- in wrapper.sh; this variable can be set in profile entries using the usual
- environment variables
-
-(04/14/08)
-*** Constant interning - this reduces the number of datasets for local values
- created, especially when constants are used in loops.
-
-(04/09/08)
-*** Swift 0.5 built from Swift SVN r1783 and CoG SVN r1962
-
-(03/26/08)
-*** String literals now use \ as escape character - for example, use \"
- to insert a quote into a string literal. Where \ was previously used
- in a string literal, it is now necessary to escape it as \\
-
-(03/25/08)
-
-*** Logging performance improvement in wrapper.sh
-
-(03/15/08)
-*** Unused lrc field in site catalog removed.
-
-*** url field in site catalog execute element is now optional.
-
-(03/13/08)
-
-*** Swift v0.4 built from Swift SVN r1718 and cog SVN r1934.
-
-(03/03/08)
-*** Stage out wrapper log files in a similar fashion to kickstart records.
-
-(02/28/08)
-*** Progress ticker on stderr every 10..60 seconds
-
-*** Store kickstart records in a subdirectory with the same base name as
- is used for log files.
-
-(02/27/08)
-*** SwiftScript source file line numbers are now passed deeper into the
- compiler in order to give location information in more error messages.
- This has resulted in a schema change for the XML intermediate form.
-
-(02/26/08)
-*** Better compile-time checking of variable usage. Use of undeclared
- variables and multiple assignments to the same variable will now result
- in a compile-time error (close to the start of a run), rather than an
- error during execution (which may be many hours later).
-
-(01/25/08)
-*** @arg now takes a second optional argument which stands for a default
- value in case the command line argument is missing.
-
-(01/14/08)
-*** Overloaded '+' operator to allow string concatenation
-
-(01/05/08)
-*** Multidimensional array handling. Can now declare arrays like this:
- file f[][];
- and refer to elements like this:
- f[1][1]
- f[1] refers to a one dimensional array, in the style of Java arrays.
-
-(12/18/07)
-*** r1214 made type specification illegal in a foreach but still recognised it
- and threw an error. This commit removes recognition of that from the
- parser entirely (as was intended to eventually happen when r1214 was made)
-
-*** Complete removal of "while" statements from the language.
- They do not seem to have a use, and haven't worked for some months
- due to their removal from the XML intermediate form.
-
-(12/07/07)
-*** Deprecated use of .dtm extension for source files. Use .swift instead.
- Support for .dtm will be dropped eventually.
-
-(12/03/07)
-*** Added -runid parameter that specifies the run identifier on the command
- line, rather than having it automatically generated. This must
- be unique for every invocation of a workflow and is used in several
- places to keep files from different executions cleanly separated. By
- default, a datestamp and random number are used to generate a run
- identifier. When using this parameter, care should be taken to ensure
- that the run ID remains unique with respect to all other run IDs that
- might be used, irrespective of (at least) expected run location,
- workflow or user.
-
-(11/23/07)
-*** new parameter, execution.retries to specify the number of time a job
- will be retried if it fails (giving a maximum of 1 + execution.retries
- attempts at execution)
-
-*** Added filesystem site catalog element. Looks like 'execution' and
- allows, for example, SSH to be used for remote filesystem access.
-
-(11/20/07)
-
-*** Many unused fields in the sites catalog, sites.xml, are now optional.
-
-*** Nested statement blocks are no longer supported. These saw little use
- and were buggy.
-
-(11/01/07)
-
-*** Concurrent mapper now generates filenames in a relatively deep directory
- hierarchy in order to reduce the number of files in each directory.
- There is a (hard-coded) parameter which determines how many files and
- subdirectories will be permitted at each level. This parameter is set
- to 25.
-
-*** move various job related directories into subdirs on site-side.
- at present, the subdirs are named by a single digit directory.
-
-(10/31/07)
-
-*** Added an external mapper to allow invoking a script to do the
- actual mapping
-
-(10/08/07)
-
-*** Added readData function
-
-(10/03/07)
-
-*** Profile names/keys in tc.data are now case insensitive
-
-(10/02/07)
-
-*** Swift 0.3 release candidate 1 built from swift svn r1312, cog r1769
-
-(09/28/07)
-
-*** Clustering is now disabled by default.
-
-*** Over the past week, a number of changes have been made to the logging
- output of Swift to make it richer. This is likely to break existing
- log processing code.
-
-(09/19/07)
-
-*** Subtle changes to the behaviour of AbstractFileMapper and its subclasses
- (primarily any of the mappers that take prefix, suffix or location
- parameters). The behaviour of prefixes and suffixes was not consistent
- between forward and reverse mapping - this has been made more consistent.
- Prefixes are now added when mapping directly as strings, rather than
- passing them through the path component mapping code first. Handling of
- separators between fields is slightly changed to accomodate this - a
- special case for the first separator is removed. Detection of the
- types of fields/indices in a path is now done using Path.Entry methods
- rather than guessing based on the first character of the field name.
-
-*** Added lots of debug logging to AbstractFileMapper
-
-(09/18/07)
-
-*** Add a basic iterate {} language construct, and an @extractint function
- that can be used with it.
-
-*** Add a @regexp language function which provides more regular expression
- functionality than is provided by @strcut. This is intended to allow
- the regular expression mapper to be removed (with uses of it replaced
- with the single file mapper and the @regexp function)
-
-(09/13/07)
-
-*** Changed lazy error default from true to false. This affect how
- Swift handles workflows when an error occurs. See the userguide
- for further details.
-
-(09/10/07)
-
-*** Removed ability to specify type of index variable in foreach loops. This
- type was never checked. Any programs which specify the type in a foreach
- statement will need a code modification to remove that type specification.
- Making such a change will not modify workflow behaviour.
-
-(09/08/07)
-
-*** Changed the way arguments are passed to the wrapper. This
- should be more resilient to broken middleware that doesn't
- properly quote arguments.
-
-(09/04/07)
-
-*** Updated wrapper to be more thorough at diagnosing problems.
-
-(08/29/07)
-
-*** Moved @arg to swiftscript namespace/package
-
-*** @filenames disappeared at some point in time. It is back.
-
-*** Fixed some issues with paths returned by fringePaths being
- parsed into the wrong things. It points out that one should
- be careful about methods typically used to get some human
- readable representation of data (such as toString()) being
- relied on to provide any kind of functional consistency.
-
-(08/28/07)
-
-*** Added throttle.score.job.factor to swift.properties (the functionality
- was previously available by manually hacking scheduler.xml)
-
-*** Throttle options now accept "off" as a valid value
-
-*** Put @arg back in
-
-*** Changes the configuration to allow loading a specific file even
- if the defaults cannot be found.
-
-*** Added PBS and dCache providers as dependencies
-
-*** Typechecks on arrays would fail (this was introduced by the type
- system changes). Fixed.
-
-*** Assignments from integer[] to float[] should not fail for type
- reasons (nor should passing an integer[] for a float[] fail the
- typecheck).
-
-(08/23/07)
-
-*** A few changes to the way types work. Whether something is an array
- or not is part of the type now.
-
-*** Cleanups to the way things get stored and mappers
-
-*** It's not necessary to manually add task handlers any more. All the
- available ones are added automatically
-
-(08/16/07)
-
-*** A different site catalog element <execution> that fills a similar role
- to the existing <jobmanager> element with a syntax more sympathetic to
- submitting via non-GRAM submission systems.
-
-(07/31/07)
-
-*** Prototype structured_regexp_mapper for performing a regexp transformation
- on the filenames of one dataset to produce another dataset with the
- same structure but different leaf filenames.
-
-(07/30/07)
-
-*** All simple types are now passed around inside of DSHandle objects,
- instead of sometimes inside DSHandles and sometimes as rawer Java
- objects.
-
-*** Swift operators now implemented in their own class, rather than using
- the Karajan implementation.
-
-*** There are now two division operators, / and %/ of which the first (/)
- returns a floating point value and %/ returns an integer value.
- The mod operator is relabelled %% instead of %.
-
-*** Mappers can now map files on remote sites (use protocol://host/path)
- for file name(s), and the engine can actually stage things in and out
- from/to arbitrary locations
-
-*** Moved some path manipulation function to Java
-
-(07/29/07)
-
-*** Use /dev/urandom by default on UNIX platforms to avoid large
- delays at start-up (and Yoya knows when else). It will lower the
- security of anything that relies on java.security.SecureRandom
- for RNG, but there is no nice way to initialize different SecureRandom
- instances with different seed generators.
-
-(07/28/07)
-
-*** Removed break; statement from the language. It was previously unimplemented
- and acted as a no-op. Any code which now does not compile can be made to
- compile by removing the break statements. No other changes should be
- necessary, and no behavioural changes should result.
-
-(07/19/07)
-
-*** Swift 0.2 release made from Swift SVN r915 and CoG SVN r1658
-
-(05/21/07)
-
-*** Add a mapper, array_mapper, which maps from a string array of filenames.
- This was requested in bug 27.
-
-(04/27/07)
-
-*** Fixed a deadlock in the remote caching code.
-
-(04/12/07)
-
-*** Marker types (that is, types with no content) can now be specified without
- an empty {} block. So what was previous written as:
-
- type imagefile {};
-
- can now be written as
-
- type imagefile;
-
-(04/11/07)
-
-*** CSV mapper will now take commas as a field separator (in addition to the
- existing behaviour of space and tab)
-
-(03/23/07)
-
-*** Fixed -resume
-
-*** Fixed code dealing with command line arguments
-
-(03/16/07)
-
-*** Labels in graphs are quoted now
-
-(03/12/07)
-
-*** Cache remote directory structure creation.
-
-(03/03/07)
-
-*** Moved job directory creation to the wrapper
-
-(03/03/07)
-
-*** The default values for mapping parameters handling below
- did not allow implementations to get the value of an
- optional parameter by checking if it's not null. This can
- now be done by specifying null as a default value.
-
-(03/02/07)
-
-*** Added log message to print full stack trace of exceptions
- that cause Swift to fail
-
-*** Better handling of default values for mapping parameters,
- and better handling of mapping parameter errors.
-
-(03/01/07)
-
-*** Fixed @arg.
-
-(02/19/07)
-
-*** Fixed a race in the cache
-
-(02/16/07)
-
-*** Fixed stdin issue (bug #19)
-
-*** Fixed an issue with vdl:cacheAndLockFile considering files for
- which the transfers failed as being actually cached.
-
-(02/14/07)
-
-*** Print usage and exit instead of a nasty exception if the input
- file does not exist (bug #15)
-
-(02/13/07)
-
-*** Ongoing parser changes
-
-*** Make sure /bin and /usr/bin are added to the path in the
- wrapper.
-
-*** When there's a NPE or CCE in an exception chain that causes the
- execution to fail, the full stack trace is printed.
-
-(02/11/07)
-
-*** SwiftScript syntax change: The function invocation (i=q)=f(1) must
- be written with the () parentheses surrounding the return variables.
- Previously, the parentheses were optional, and the same could be written
- as i=q=f(1); This is no longer the case.
-
-(02/10/07)
-
-*** On-going parser changes primarily aimed at producing more user-friendly
- syntax errors but also including these syntax changes:
- - parser no longer recognises vestigial for(;;) construct
- - parser allows type declarations and other top level statements
- (aside from namespace declarations) to be mixed together. Previously,
- all type declarations had to occur before other statements.
-
-(02/09/07)
-
-*** Reorganized logging. Log always goes to a file. Console
- gets little stuff unless -verbose or -debug are used.
-
-*** Use a unified run id which is used as a prefix for the
- log, the restart log, the provenance graph, and the
- remote run directories
-
-*** Hmm. Did I mess up the dates on the changelog?
-
-*** Added default properties that work in case
- swift/etc/swift.properties is missing some.
-
-(02/08/07)
-
-*** Added option to always transfer Kickstart records
-
-*** Improved error handling for compilation of SwiftScript code
-
-(02/07/07)
-
-*** Added Kickstart support (optional)
-
-*** Added support for GLOBUS profiles in the site catalog
-
-*** Updated error translations to include a pointer to the
- Kickstart info if available
-
-(02/05/07)
-
-*** Renamed a few things to swift, including:
- - vdl2.properties -> swift.properties
- - ~/.vdl2 -> ~/.swift (the old config is still read though)
- - the log4j "class" for things in Karajan (vdl2 -> swift)
- - vdlrun -> swift
-
-*** swift (vdlrun) will also work with .dtm files now, and compile
- them on the fly.
-
-*** Most of the throttle parameters are now settable through
- swift.properties
-
-*** Made some preliminary (and incomplete) changes towards integrating
- kickstart.
-
-(02/02/07)
-
-*** Renamed a few mappers to better reflect the mapper functionalities.
-
-*** Added warnings mechanism
-
-*** No more Karajan stack traces in errors (only simple chained messages).
- Those stack traces belong to the logging domain
-
-*** Capture cleanup problems and print them as warnings
-
-*** Added a few more error translation
-
-(01/30/07)
-
-*** Fixed vdlc to return a non-zero exit code when text->xml conversion
- fails.
-
-(01/28/07)
-
-*** Added support for non-standard attributes on the GLOBUS
- prefix in tc.data.
-
-(01/18/07)
-
-*** Fixed an issue with dependent mappers and arrays
-
-*** Fixed the concurrent mapper
-
-(01/12/07)
-
-*** Added mapping dependencies.
-
-(01/06/07)
-
-*** Added a cache around the Pegasus TC file implementation.
-
-*** Added vdl:log as a wrapper around log4j.
-
-*** Fixed an issue in the cleanup with the same file being
- deleted multiple times concurrently.
-
-
-(01/04/07)
-
-*** Added a simple clustering mechanism for small jobs.
-
-*** Added a "-dryrun" argument to vdlrun. If graph generation
- is enabled, this should produce a graph without running
- any actual jobs.
-
-*** Fixed an issue with caching where a file could have
- been purged more than once
-
-(01/01/07)
-
-*** Modified AbstractFileMapper to expose local variables as protected
-
-*** Added a RegularExpressionMapper to do file name conversion based on regexp
-
-*** Added a DirectoryMapper to map a list of files from a directory as an array
-
-*** Added a CSVMapper to map a CSV (Comma-Separated Values) file into a logical structure
-
-(12/30/06)
-
-*** A new vdlrun that is specific to VDL2.
-
-*** Added an error translator (based on regular expressions)
- that can be used to provide improved error messages
-
-*** Added a lazy errors mode in which errors in application
- execution are aggregated and reported only once at the
- end
-
-*** Added provenance graph generation
-
-*** Added a control mechanism for managing remote storage
- usage
-
-*** Cleanups
-
-
-------------------------- 0 rc 3 ---------------------------
-
-(12/28/06)
-
-*** The heuristics (in CoG) used to detect if a task can ever
- run (due to constraints as opposed to temporary load
- conditions) were broken and the issue was fixed.
-
-(12/23/06)
-------------------------- 0 rc 2 ---------------------------
-
-*** svn copy is broken so I'm noting the revision here: 115
-
-(12/14/06)
-
-*** Fixed the default mapper to be concurrent_mapper
-
-*** Added passing of tc.data profiles to the job (e.g
- user-defined environment variables, jobtype, count, etc.)
-
-------------------------- 0 rc 1 ---------------------------
-(12/07/06)
-
-*** Added dynamic loading of the TC file
-
-*** Added cleanup which is done at the end of the
- whole run
-
-(12/06/06)
-
-*** Moved to CI SVN
-
-*** Added tests directory
-
-(11/22/06)
-
-*** Changed to a pure Karajan adaptive scheduler instead
- of a Pegasus site selector
-
-*** Added "client-side-caching" of remote files
-
-*** Added a basic wrapper to capture exit codes and do other
- work before and after executing the job
-
-
Added: tags/swift-0.94.1/RELEASENOTES.txt
===================================================================
--- tags/swift-0.94.1/RELEASENOTES.txt (rev 0)
+++ tags/swift-0.94.1/RELEASENOTES.txt 2013-09-30 17:15:50 UTC (rev 7115)
@@ -0,0 +1,248 @@
+Swift 0.94.1 Release Notes
+==========================
+
+Iterate changes
+---------------
+To behavior of iterate has changed from 0.93 to 0.94.1. The until() clause of
+the iterate statement now causes the loop to stop as soon as the until clause
+returns "true", instead of doing one additional iteration when the clause
+turns true.
+
+For example, iterate code in 0.93 might say:
+-----
+iterate i {
+...
+} until (i == 9);
+-----
+To achieve the same 10 executions in 0.94.1, you must say:
+-----
+iterate i {
+...
+} until (i==10);
+-----
+
+More information about iterate can be found in the
+http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_iterate[iterate section of the Swift User Guide].
+
+Coaster walltimes
+-----------------
+Walltimes are now strictly enforced by coasters.
+
+There are two coaster properties that affect how walltimes are handled. They
+are maxtime and maxwalltime.
+
+Maxtime is the maximum amount of time, in seconds, that a coaster block can run.
+This is defined in sites.xml.
+
+.Setting a maxTime of 30 minutes
+-----
+<profile namespace="globus" key="maxTime">1800</profile>
+-----
+
+The maxwalltime property defines the maximum amount of time that any given
+application may run (in hh:mm:ss format)
+
+.Setting a maxWallTime of 5 minutes
+-----
+<profile namespace="globus" key="maxWallTime">00:05:00</profile>
+-----
+
+An application may run up to the the limit defined in maxwalltime, and are
+packed into coasters slots that run for maxtime seconds.
+
+
+In versions of Swift prior to 0.94, if an application exceeded its specified
+maxwalltime, execution would continue. However, if the application exceeded
+its maxwalltime, the queuing system could kill the worker. The resulting error
+message was not always very clear.
+
+In Swift 0.94.1, coaster workers enforce the user-specified
+maxwalltime. If an application exceeds its maxwalltime, the coaster worker
+will not allow it to continue, but terminate it and report the error.
+
+The error message will look something like this:
+-----
+Execution failed:
+ Exception in hostsnsleep:
+ Arguments: [30]
+ Host: midway
+ Directory: hostsnsleep-20130502-2009-t8m0l3hb/jobs/n/hostsnsleep-n55v8x8l
+ stderr.txt:
+ stdout.txt: midway009
+Caused by:
+ Walltime exceeded
+org.globus.cog.abstraction.impl.common.execution.JobException: Walltime exceeded (exit code: 513)
+ at org.globus.cog.abstraction.coaster.service.local.JobStatusHandler.requestComplete(JobStatusHandler.java:38)
+ at org.globus.cog.karajan.workflow.service.handlers.RequestHandler.receiveCompleted(RequestHandler.java:88)
+ at org.globus.cog.karajan.workflow.service.channels.AbstractKarajanChannel.handleRequest(AbstractKarajanChannel.java:518)
+ at org.globus.cog.karajan.workflow.service.channels.AbstractStreamKarajanChannel.stepNIO(AbstractStreamKarajanChannel.java:238)
+ at org.globus.cog.karajan.workflow.service.channels.NIOMultiplexer.loop(NIOMultiplexer.java:97)
+ at org.globus.cog.karajan.workflow.service.channels.NIOMultiplexer.run(NIOMultiplexer.java:56)
+ (exit code: 513)
+ hostsnsleep, hostsnsleep.swift, line 13
+-----
+
+TIP: Maxwalltime may also be define on a per application level in your tc file
+(http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_globus_namespace[Globus Namespace section, Swift User Guide]).
+
+Camel Case Functions
+--------------------
+To make things more consistent, many swift functions are being renamed
+to use camel case naming. For example, @toint is now @toInt. The previous
+naming conventions will still work in 0.94, but may not in future releases.
+If you try to use a function which has been renamed, you will see deprecated
+warnings like this:
+-----
+Warning: Function toint is deprecated, at 2
+-----
+Below is a list of functions with updated names.
+
+[options="header", width="25%"]
+|===========================
+| Old Name | New Name
+| @extractint | @extractInt
+| @toint | @toInt
+| @tofloat | @toFloat
+| @tostring | @toString
+|===========================
+
+
+Associative Arrays
+------------------
+Associative arrays have been added. By default, array keys are integers, but in
+0.94.1 other primitive types are also allowed as array keys. More details and
+examples can be found in the
+http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_associative_arrays[associate array section of the Swift User Guide].
+
+Dynamic profiles
+----------------
+Many settings formerly only definable in sites.xml can now
+be set on a per-app basis. This can make things easier when running
+multiple apps that have different requirements for settings like processors
+per node and walltime.
+
+More information and examples can be found in the
+http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_dynamic_profiles[dynamic profiles section of the Swift User Guide]
+
+SSH command line provider
+-------------------------
+Added a new ssh command line provider. Previously ssh support was done by
+creating a file called ~/.ssh/auth.defaults. The ssh command line provider
+is more flexible and doesn't require this step. ssh-cl allows you to use
+your existing SSH agents. You can use ssh-cl to run remotely by adding
+something like this to your sites.xml:
+
+-----
+ <execution provider="coaster" url="my.host.uchicago.edu" jobmanager="ssh-cl:pbs"/>
+-----
+
+More information about this can be found in the
+http://www.ci.uchicago.edu/swift/guides/release-0.94/siteguide/siteguide.html#_ssh_cl[Site
+Guide].
+
+Coaster provider staging improvements
+-------------------------------------
+Coasters provides a mechanism for transferring files between hosts, called
+coaster provider staging. Coaster provider staging is explained in more
+detail in http://www.ci.uchicago.edu/swift/papers/UCC-coasters.pdf[this
+research paper].
+
+In 0.94 there have been many fixes and improvements to make coaster
+provider staging more reliable.
+
+Added support for the Slurm scheduler
+-------------------------------------
+Swift 0.94 includes support for Slurm - an open source job scheduler used on
+many supercomputers and computer clusters.
+
+The http://www.ci.uchicago.edu/swift/guides/trunk/siteguide/siteguide.html#_midway_x86_cluster[Swift Site Guide entry for Midway]
+has an entry containing information about how to run Swift on a specific Slurm
+cluster at the University of Chicago called Midway. By changing queue names
+and project names, this information could be used to run on any Slurm cluster.
+
+Added support for the LSF scheduler
+-----------------------------------
+Swift 0.94 includes support for the LSF (Load Sharing Facility) scheduler. An
+example sites.xml can be found in the
+http://www.ci.uchicago.edu/swift/guides/release-0.94/siteguide/siteguide.html#_geyser_and_caldera_x86_clusters[Swift Site Guide entry for Geyser].
+
+Condor Improvements
+-------------------
+Some improvements have been made to the condor provider. For condor systems
+without a shared filesystem, Swift can now use Condor's built-in file transfer
+mechanisms to transfer the scripts it needs to starts jobs. To do this, you can
+define the jobtype to "non-shared" in sites.xml.
+
+-----
+<profile namespace="globus" key="jobType">nonshared</profile>
+-----
+
+An example setup that uses this style of configuration can be found in the
+http://www.ci.uchicago.edu/swift/guides/release-0.94/siteguide/siteguide.html#_uc3_x86_cluster[Swift siteguide entry for the UC3 cluster].
+
+Textual User Interface
+----------------------
+Fixes for the textual user interface (TUI). Adding the -tui option to the swift
+command line allows you to monitor progress in a curses based menu. For example:
+-----
+swift -sites.file sites.xml -tc.file tc.data -tui myscript.swift
+-----
+
+A brief example of this can be found at http://www.ci.uchicago.edu/~davidk/modis.ogv.
+
+ at java
+-----
+Added the ability to call Java methods within swift using @java. For example:
+-----
+float f = @java("java.lang.Math", "sin", 0.5);
+-----
+More information can be found in the
+http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_java[Swift User Guide entry on @java].
+
+Hang Checker
+------------
+Added a hang checker that provides the user with more information about
+potential hangs. Here is one example of a situation that may cause Swift
+to hang:
+
+-----
+int a;
+int b;
+
+a = b;
+b = a;
+-----
+
+When this script is run, you will now see a more detailed message explaining
+what's happening.
+
+-----
+Dependency loop found:
+ b (declared on line 2) is needed by:
+ a = b, test.swift, line 5
+ which produces a (declared on line 1)
+
+ a (declared on line 1) is needed by:
+ b = a, test.swift, line 6
+ which produces b (declared on line 2)
+-----
+
+ at strjoin
+--------
+A function called @strjoin function for joining strings. More information and examples are
+in the
+http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_strjoin[Swift User Guide section on @strjoin].
+
+Swift Userhome
+--------------
+Swift creates some files it needs by default in your home directory. If
+your home filesystem is not available on the worker nodes of a cluster you are
+trying to use, this can cause tasks to fail. Setting the environment variable
+SWIFT_USERHOME to the location of a shared filesystem will get around this
+restriction.
+
+MPI Improvements
+----------------
+Swift 0.94 has improved MPI support and integration. Details about how to
+run MPI jobs in Swift can be found in the
+http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_launching_mpi_jobs_from_swift[the MPI section of the Swift User Guide].
More information about the Swift-commit
mailing list