[Swift-devel] 0.94 release note draft

[David Kelly]

Good point, I'll rephrase that to make it clearer. I believe it's handled by _swiftwrap.wrapperstaging. 

----- Original Message -----

> From: "Ketan Maheshwari" <kcm92 at cornell.edu>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "Michael Wilde" <wilde at mcs.anl.gov>, "Swift Devel"
> <swift-devel at ci.uchicago.edu>
> Sent: Friday, March 1, 2013 9:45:27 AM
> Subject: Re: [Swift-devel] 0.94 release note draft

> One comment about the 'wrapper staging' feature: my understanding was
> that wrapper staging means the files are staged by the swift
> wrapper: _swiftwrap and not an external wrapper.

> Correct me if I am wrong.

> Regards,
> Ketan

> On Fri, Mar 1, 2013 at 8:06 AM, David Kelly < davidk at ci.uchicago.edu
> > wrote:

> > Here is the list, based mostly on what I could find in the svn
> > logs.
> 

> > - To behavior of iterate has changed from 0.93 to 0.94. If you have
> > scripts
> 
> > that use iterate, please read
> 
> > http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_iterate
> 

> > - Walltimes are more strictly enforced by coasters. Previous to
> > Swift
> > version
> 
> > 0.94, if an application run with coasters would exceed its
> > specified
> 
> > maxwalltime, it would be allowed to continue to execute. However,
> > if
> > this
> 
> > would cause the worker on which the application was running to
> > exceed
> > its
> 
> > maxwalltime, the queuing system would kill the worker. The
> > resulting
> > error
> 
> > message was not always very clear. Since version 0.94 coaster
> > workers
> > enforce
> 
> > the user-specified maxwalltime. If an application exceeds its
> > maxwalltime,
> 
> > the coaster worker will not allow it to continue, but terminate it
> > and report
> 
> > the error.
> 

> > - Swift will now use camel case for functions, for example, @toInt
> > instead of
> 
> > @toint. The previous naming convention will still work, but you may
> > see
> 
> > deprecated warnings.
> 

> > - Associative arrays have been added. More details and examples can
> > be found at
> 
> > http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_associative_arrays
> 

> > - Dynamic profiles. Many settings formerly only definable in
> > sites.xml can now
> 
> > be set on a per-app basis. This can make things easier when running
> 
> > multiple apps that have different requirements for settings like
> > processors
> 
> > per node and wallitme.
> 
> > http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_dynamic_profiles
> 

> > - Added a new ssh command line provider. Previously ssh support was
> > done by
> 
> > creating a file called ~/.ssh/auth.defaults. The ssh command line
> > provider
> 
> > is more flexible and doesn't require this step. ssh-cl allows you
> > to
> > use
> 
> > SSH agents. You can use ssh-cl by adding something like this to
> > your
> > sites.xml:
> 

> > <execution provider="coaster" url=" my.host.uchicago.edu "
> > jobmanager="ssh-cl:pbs"/>
> 

> > - Many fixes and improvements to improve the reliability and
> > performance of
> 
> > coaster provider staging.
> 

> > - Added support for the Slurm scheduler
> 

> > - Added support for the LSF scheduler
> 

> > - Improvements to condor provider (non-shared jobtype and more
> > flexibility
> 
> > to define what gets added to the submit script).
> 

> > - Fixes for the textual user interface (TUI). Adding the -tui
> > option
> > to the swift
> 
> > command line allows you to monitor progress in a curses based menu.
> > A
> > brief
> 
> > example of this can be found at
> > http://www.ci.uchicago.edu/~davidk/modis.ogv .
> 

> > - Added the ability to call Java methods within swift using @java.
> > For example:
> 
> > float f = @java("java.lang.Math", "sin", 0.5);
> 
> > http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_java
> 

> > - Added a hang checker that provides the user with more information
> > about
> 
> > potential hangs
> 

> > - @strjoin function for joining strings.
> 
> > http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_strjoin
> 

> > - If you have a requirement that a command get run on the worker
> > node
> > before
> 
> > processing any work, worker.pl will now execute commands stored in
> > the
> 
> > environment variable $WORKER_INIT_CMD
> 

> > - Use $SWIFT_USERHOME to determine where to swift should create
> > some
> > of its
> 
> > required files. This defaults to $HOME, but this may cause problems
> > in some
> 
> > situations where $HOME is not accessable on worker nodes.
> 

> > - Experimental "wrapper staging" feature that delegates file
> > staging
> > to an
> 
> > external wrapper script.
> 

> > - Various improvements to the way that Swift runs MPI jobs.
> 

> > - Better OSG integration/support using GlideinWMS.
> 

> > > From: "Michael Wilde" < wilde at mcs.anl.gov >
> > 
> 
> > > To: "David Kelly" < davidk at ci.uchicago.edu >
> > 
> 
> > > Cc: "Swift Devel" < swift-devel at ci.uchicago.edu >
> > 
> 
> > > Sent: Thursday, February 14, 2013 1:08:56 PM
> > 
> 
> > > Subject: Fwd: 0.94 release note draft
> > 
> 

> > > Some notes toward an 0.94 release notes document.
> > 
> 

> > > There's a longer list, I think in an IM chat transcript, that we
> > > need
> > > to incorporate.
> > 
> 

> > > Please send additional items to this thread for David to
> > > integrate.
> > 
> 

> > > Thanks,
> > 
> 

> > > - Mike
> > 
> 

> > > ----- Forwarded Message -----
> > 
> 
> > > From: "David Kelly" < davidk at ci.uchicago.edu >
> > 
> 
> > > To: "Michael Wilde" < wilde at mcs.anl.gov >
> > 
> 
> > > Sent: Thursday, January 24, 2013 11:06:18 PM
> > 
> 
> > > Subject: Re: 0.94 release note draft
> > 
> 

> > > Mike,
> > 
> 

> > > I just have the quick notes I took from our meeting. These
> > > combined
> > > with your emails are all the changes that I'm aware of at this
> > > point.
> > 
> 

> > > Iterate differences
> > 
> 
> > > Walltime hard limit with coasters
> > 
> 
> > > Associate arrays changes
> > 
> 
> > > Tracebacker
> > 
> 
> > > Coaster changes / parameters
> > 
> 
> > > Slurm and LSF providers
> > 
> 
> > > Condor provider changes
> > 
> 
> > > ssh-cl
> > 
> 
> > > TUI
> > 
> 
> > > hang checker
> > 
> 
> > > @functions (strjoin, and possibly others)
> > 
> 
> > > Dynamic profiles
> > 
> 
> > > Wrapper staging
> > 
> 
> > > Pass-thru (PBS attributes)
> > 
> 
> > > MPI support
> > 
> 

> > > ----- Original Message -----
> > 
> 
> > > > From: "Michael Wilde" < wilde at mcs.anl.gov >
> > 
> 
> > > > To: "David Kelly" < davidk at ci.uchicago.edu >
> > 
> 
> > > > Sent: Thursday, January 24, 2013 7:56:54 PM
> > 
> 
> > > > Subject: 0.94 release note draft
> > 
> 
> > > >
> > 
> 
> > > >
> > 
> 
> > > > Hi David,
> > 
> 
> > > >
> > 
> 
> > > > I recall I sent you a few batches of line items to list in 0.94
> > 
> 
> > > > release notes. Did you gather those somewhere where I can
> > > > review
> > 
> 
> > > > them? (Need them for a status report)
> > 
> 
> > > >
> > 
> 
> > > > Thanks,
> > 
> 
> > > >
> > 
> 
> > > > - Mike
> > 
> 
> > > >
> > 
> 
> > > >
> > 
> 
> > > > --
> > 
> 
> > > > Michael Wilde
> > 
> 
> > > > Computation Institute, University of Chicago
> > 
> 
> > > > Mathematics and Computer Science Division
> > 
> 
> > > > Argonne National Laboratory
> > 
> 
> > > >
> > 
> 
> > > >
> > 
> 

> > _______________________________________________
> 
> > Swift-devel mailing list
> 
> > Swift-devel at ci.uchicago.edu
> 
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20130301/bc7cd95f/attachment.html>