[Swift-devel] 0.94 release note draft

[David Kelly]

Here is the list, based mostly on what I could find in the svn logs. 



- To behavior of iterate has changed from 0.93 to 0.94. If you have scripts 
that use iterate, please read 
http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_iterate 


- Walltimes are more strictly enforced by coasters. Previous to Swift version 
0.94, if an application run with coasters would exceed its specified 
maxwalltime, it would be allowed to continue to execute. However, if this 
would cause the worker on which the application was running to exceed its 
maxwalltime, the queuing system would kill the worker. The resulting error 
message was not always very clear. Since version 0.94 coaster workers enforce 
the user-specified maxwalltime. If an application exceeds its maxwalltime, 
the coaster worker will not allow it to continue, but terminate it and report 
the error. 


- Swift will now use camel case for functions, for example, @toInt instead of 
@toint. The previous naming convention will still work, but you may see 
deprecated warnings. 


- Associative arrays have been added. More details and examples can be found at 
http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_associative_arrays 


- Dynamic profiles. Many settings formerly only definable in sites.xml can now 
be set on a per-app basis. This can make things easier when running 
multiple apps that have different requirements for settings like processors 
per node and wallitme. 
http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_dynamic_profiles 


- Added a new ssh command line provider. Previously ssh support was done by 
creating a file called ~/.ssh/auth.defaults. The ssh command line provider 
is more flexible and doesn't require this step. ssh-cl allows you to use 
SSH agents. You can use ssh-cl by adding something like this to your sites.xml: 


<execution provider="coaster" url="my.host.uchicago.edu" jobmanager="ssh-cl:pbs"/> 


- Many fixes and improvements to improve the reliability and performance of 
coaster provider staging. 


- Added support for the Slurm scheduler 


- Added support for the LSF scheduler 


- Improvements to condor provider (non-shared jobtype and more flexibility 
to define what gets added to the submit script). 


- Fixes for the textual user interface (TUI). Adding the -tui option to the swift 
command line allows you to monitor progress in a curses based menu. A brief 
example of this can be found at http://www.ci.uchicago.edu/~davidk/modis.ogv. 


- Added the ability to call Java methods within swift using @java. For example: 
float f = @java("java.lang.Math", "sin", 0.5); 
http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_java 


- Added a hang checker that provides the user with more information about 
potential hangs 


- @strjoin function for joining strings. 
http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_strjoin 


- If you have a requirement that a command get run on the worker node before 
processing any work, worker.pl will now execute commands stored in the 
environment variable $WORKER_INIT_CMD 


- Use $SWIFT_USERHOME to determine where to swift should create some of its 
required files. This defaults to $HOME, but this may cause problems in some 
situations where $HOME is not accessable on worker nodes. 


- Experimental "wrapper staging" feature that delegates file staging to an 
external wrapper script. 


- Various improvements to the way that Swift runs MPI jobs. 


- Better OSG integration/support using GlideinWMS. 


----- Original Message -----


From: "Michael Wilde" <wilde at mcs.anl.gov> 
To: "David Kelly" <davidk at ci.uchicago.edu> 
Cc: "Swift Devel" <swift-devel at ci.uchicago.edu> 
Sent: Thursday, February 14, 2013 1:08:56 PM 
Subject: Fwd: 0.94 release note draft 


Some notes toward an 0.94 release notes document. 

There's a longer list, I think in an IM chat transcript, that we need to incorporate. 

Please send additional items to this thread for David to integrate. 

Thanks, 

- Mike 

----- Forwarded Message ----- 
From: "David Kelly" <davidk at ci.uchicago.edu> 
To: "Michael Wilde" <wilde at mcs.anl.gov> 
Sent: Thursday, January 24, 2013 11:06:18 PM 
Subject: Re: 0.94 release note draft 

Mike, 

I just have the quick notes I took from our meeting. These combined with your emails are all the changes that I'm aware of at this point. 

Iterate differences 
Walltime hard limit with coasters 
Associate arrays changes 
Tracebacker 
Coaster changes / parameters 
Slurm and LSF providers 
Condor provider changes 
ssh-cl 
TUI 
hang checker 
@functions (strjoin, and possibly others) 
Dynamic profiles 
Wrapper staging 
Pass-thru (PBS attributes) 
MPI support 


----- Original Message ----- 
> From: "Michael Wilde" <wilde at mcs.anl.gov> 
> To: "David Kelly" <davidk at ci.uchicago.edu> 
> Sent: Thursday, January 24, 2013 7:56:54 PM 
> Subject: 0.94 release note draft 
> 
> 
> Hi David, 
> 
> I recall I sent you a few batches of line items to list in 0.94 
> release notes. Did you gather those somewhere where I can review 
> them? (Need them for a status report) 
> 
> Thanks, 
> 
> - Mike 
> 
> 
> -- 
> Michael Wilde 
> Computation Institute, University of Chicago 
> Mathematics and Computer Science Division 
> Argonne National Laboratory 
> 
> 



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20130301/9b72fff3/attachment.html>