<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: times new roman,new york,times,serif; font-size: 12pt; color: #000000'><font face="times new roman, new york, times, serif">Good point, I'll rephrase that to make it clearer. I believe it's handled by </font><span style="font-size: medium; background-color: rgb(255, 255, 255);"><font face="times new roman, new york, times, serif">_swiftwrap.wrapperstaging.</font></span><br><br><hr id="zwchr" style="font-family: 'times new roman', 'new york', times, serif;"><blockquote style="font-family: Helvetica, Arial, sans-serif; border-left-width: 2px; border-left-style: solid; border-left-color: rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; color: rgb(0, 0, 0); font-weight: normal; font-style: normal; text-decoration: none; font-size: 12pt;"><b>From: </b>"Ketan Maheshwari" <kcm92@cornell.edu><br><b>To: </b>"David Kelly" <davidk@ci.uchicago.edu><br><b>Cc: </b>"Michael Wilde" <wilde@mcs.anl.gov>, "Swift Devel" <swift-devel@ci.uchicago.edu><br><b>Sent: </b>Friday, March 1, 2013 9:45:27 AM<br><b>Subject: </b>Re: [Swift-devel] 0.94 release note draft<br><br><div dir="ltr">One comment about the 'wrapper staging' feature: my understanding was that wrapper staging means the files are staged by the swift wrapper: _swiftwrap and not an external wrapper.<div><br></div><div style="">
Correct me if I am wrong.</div><div style=""><br></div><div style="">Regards,</div><div style="">Ketan</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Mar 1, 2013 at 8:06 AM, David Kelly <span dir="ltr"><<a href="mailto:davidk@ci.uchicago.edu" target="_blank">davidk@ci.uchicago.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-size:12pt;font-family:times new roman,new york,times,serif"><font face="times new roman, new york, times, serif" size="3">Here is the list, based mostly on what I could find in the svn logs.</font><div style="font-size:12pt;font-family:'times new roman','new york',times,serif">
<br></div><div><div><font face="times new roman, new york, times, serif">- To behavior of iterate has changed from 0.93 to 0.94. If you have scripts </font></div><div><font face="times new roman, new york, times, serif"> that use iterate, please read </font></div>
<div><font face="times new roman, new york, times, serif"> <a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_iterate" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_iterate</a> </font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Walltimes are more strictly enforced by coasters. Previous to Swift version </font></div>
<div><font face="times new roman, new york, times, serif"> 0.94, if an application run with coasters would exceed its specified </font></div><div><font face="times new roman, new york, times, serif"> maxwalltime, it would be allowed to continue to execute. However, if this </font></div>
<div><font face="times new roman, new york, times, serif"> would cause the worker on which the application was running to exceed its </font></div><div><font face="times new roman, new york, times, serif"> maxwalltime, the queuing system would kill the worker. The resulting error </font></div>
<div><font face="times new roman, new york, times, serif"> message was not always very clear. Since version 0.94 coaster workers enforce</font></div><div><font face="times new roman, new york, times, serif"> the user-specified maxwalltime. If an application exceeds its maxwalltime, </font></div>
<div><font face="times new roman, new york, times, serif"> the coaster worker will not allow it to continue, but terminate it and report</font></div><div><font face="times new roman, new york, times, serif"> the error.</font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Swift will now use camel case for functions, for example, @toInt instead of </font></div>
<div><font face="times new roman, new york, times, serif"> @toint. The previous naming convention will still work, but you may see </font></div><div><font face="times new roman, new york, times, serif"> deprecated warnings.</font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Associative arrays have been added. More details and examples can be found at </font></div>
<div><font face="times new roman, new york, times, serif"> <a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_associative_arrays" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_associative_arrays</a></font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Dynamic profiles. Many settings formerly only definable in sites.xml can now</font></div>
<div><font face="times new roman, new york, times, serif"> be set on a per-app basis. This can make things easier when running</font></div><div><font face="times new roman, new york, times, serif"> multiple apps that have different requirements for settings like processors</font></div>
<div><font face="times new roman, new york, times, serif"> per node and wallitme. </font></div><div><font face="times new roman, new york, times, serif"> <a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_dynamic_profiles" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_dynamic_profiles</a></font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Added a new ssh command line provider. Previously ssh support was done by </font></div>
<div><font face="times new roman, new york, times, serif"> creating a file called ~/.ssh/auth.defaults. The ssh command line provider</font></div><div><font face="times new roman, new york, times, serif"> is more flexible and doesn't require this step. ssh-cl allows you to use</font></div>
<div><font face="times new roman, new york, times, serif"> SSH agents. You can use ssh-cl by adding something like this to your sites.xml:</font></div><div><font face="times new roman, new york, times, serif"><br></font></div>
<div><font face="times new roman, new york, times, serif"> <execution provider="coaster" url="<a href="http://my.host.uchicago.edu" target="_blank">my.host.uchicago.edu</a>" jobmanager="ssh-cl:pbs"/></font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Many fixes and improvements to improve the reliability and performance of</font></div>
<div><font face="times new roman, new york, times, serif"> coaster provider staging.</font></div><div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Added support for the Slurm scheduler</font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Added support for the LSF scheduler</font></div><div><font face="times new roman, new york, times, serif"><br>
</font></div><div><font face="times new roman, new york, times, serif">- Improvements to condor provider (non-shared jobtype and more flexibility </font></div><div><font face="times new roman, new york, times, serif"> to define what gets added to the submit script).</font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Fixes for the textual user interface (TUI). Adding the -tui option to the swift </font></div>
<div><font face="times new roman, new york, times, serif"> command line allows you to monitor progress in a curses based menu. A brief</font></div><div><font face="times new roman, new york, times, serif"> example of this can be found at <a href="http://www.ci.uchicago.edu/~davidk/modis.ogv" target="_blank">http://www.ci.uchicago.edu/~davidk/modis.ogv</a>.</font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Added the ability to call Java methods within swift using @java. For example:</font></div>
<div><font face="times new roman, new york, times, serif"> float f = @java("java.lang.Math", "sin", 0.5);</font></div><div><font face="times new roman, new york, times, serif"> <a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_java" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_java</a></font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Added a hang checker that provides the user with more information about</font></div><div>
<font face="times new roman, new york, times, serif"> potential hangs</font></div><div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- @strjoin function for joining strings.</font></div>
<div><font face="times new roman, new york, times, serif"> <a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_strjoin" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_strjoin</a> </font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- If you have a requirement that a command get run on the worker node before </font></div>
<div><font face="times new roman, new york, times, serif"> processing any work, <a href="http://worker.pl" target="_blank">worker.pl</a> will now execute commands stored in the</font></div><div><font face="times new roman, new york, times, serif"> environment variable $WORKER_INIT_CMD</font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Use $SWIFT_USERHOME to determine where to swift should create some of its </font></div>
<div><font face="times new roman, new york, times, serif"> required files. This defaults to $HOME, but this may cause problems in some </font></div><div><font face="times new roman, new york, times, serif"> situations where $HOME is not accessable on worker nodes.</font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Experimental "wrapper staging" feature that delegates file staging to an</font></div>
<div><font face="times new roman, new york, times, serif"> external wrapper script.</font></div><div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Various improvements to the way that Swift runs MPI jobs.</font></div>
<div><font face="times new roman, new york, times, serif"><br></font></div><div><font face="times new roman, new york, times, serif">- Better OSG integration/support using GlideinWMS.</font></div><div style="font-size:12pt;font-family:'times new roman','new york',times,serif">
<br></div><br><hr style="font-size:12pt;font-family:'times new roman','new york',times,serif"><blockquote style="padding-left:5px;border-left-color:rgb(16,16,255);border-left-style:solid;font-style:normal;font-size:12pt;margin-left:5px;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-weight:normal;border-left-width:2px">
<div class="im"><b>From: </b>"Michael Wilde" <<a href="mailto:wilde@mcs.anl.gov" target="_blank">wilde@mcs.anl.gov</a>><br><b>To: </b>"David Kelly" <<a href="mailto:davidk@ci.uchicago.edu" target="_blank">davidk@ci.uchicago.edu</a>><br>
</div><b>Cc: </b>"Swift Devel" <<a href="mailto:swift-devel@ci.uchicago.edu" target="_blank">swift-devel@ci.uchicago.edu</a>><br><b>Sent: </b>Thursday, February 14, 2013 1:08:56 PM<br><b>Subject: </b>Fwd: 0.94 release note draft<div>
<div class="h5"><br><br><br>Some notes toward an 0.94 release notes document.<br><br>There's a longer list, I think in an IM chat transcript, that we need to incorporate.<br><br>Please send additional items to this thread for David to integrate.<br>
<br>Thanks,<br><br>- Mike<br><br>----- Forwarded Message -----<br>From: "David Kelly" <<a href="mailto:davidk@ci.uchicago.edu" target="_blank">davidk@ci.uchicago.edu</a>><br>To: "Michael Wilde" <<a href="mailto:wilde@mcs.anl.gov" target="_blank">wilde@mcs.anl.gov</a>><br>
Sent: Thursday, January 24, 2013 11:06:18 PM<br>Subject: Re: 0.94 release note draft<br><br>Mike,<br><br>I just have the quick notes I took from our meeting. These combined with your emails are all the changes that I'm aware of at this point.<br>
<br> Iterate differences<br> Walltime hard limit with coasters<br> Associate arrays changes<br> Tracebacker<br> Coaster changes / parameters<br>
Slurm and LSF providers<br> Condor provider changes<br> ssh-cl<br> TUI<br> hang checker<br> @functions (strjoin, and possibly others)<br>
Dynamic profiles<br> Wrapper staging<br> Pass-thru (PBS attributes)<br> MPI support<br><br><br>----- Original Message -----<br>> From: "Michael Wilde" <<a href="mailto:wilde@mcs.anl.gov" target="_blank">wilde@mcs.anl.gov</a>><br>
> To: "David Kelly" <<a href="mailto:davidk@ci.uchicago.edu" target="_blank">davidk@ci.uchicago.edu</a>><br>> Sent: Thursday, January 24, 2013 7:56:54 PM<br>> Subject: 0.94 release note draft<br>> <br>
> <br>> Hi David,<br>> <br>> I recall I sent you a few batches of line items to list in 0.94<br>> release notes. Did you gather those somewhere where I can review<br>> them? (Need them for a status report)<br>
> <br>> Thanks,<br>> <br>> - Mike<br>> <br>> <br>> --<br>> Michael Wilde<br>> Computation Institute, University of Chicago<br>> Mathematics and Computer Science Division<br>> Argonne National Laboratory<br>
> <br>> <br></div></div></blockquote><br></div></div></div><br>_______________________________________________<br>
Swift-devel mailing list<br>
<a href="mailto:Swift-devel@ci.uchicago.edu" target="_blank">Swift-devel@ci.uchicago.edu</a><br>
<a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>
<br></blockquote></div><br></div>
</blockquote><br></div></body></html>