<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: times new roman,new york,times,serif; font-size: 12pt; color: #000000'><font face="times new roman, new york, times, serif" size="3">Lorenzo,</font><div style="color: rgb(0, 0, 0); font-family: 'times new roman', 'new york', times, serif; font-size: 12pt;"><br></div><div style="color: rgb(0, 0, 0); font-family: 'times new roman', 'new york', times, serif; font-size: 12pt;">That seems reasonable to me. I haven't used the faster branch myself much at this point, so it's probably best to continue talking with Mihael and Mike to come up with a good strategy for how to approach those issues. But in terms of the swift versions/modules, I have put together the three latest builds for you at:</div><div style="color: rgb(0, 0, 0); font-family: 'times new roman', 'new york', times, serif; font-size: 12pt;"><br></div><div><font face="times new roman, new york, times, serif">/home/davidk/swift-0.94-03012013/cog/modules/swift/dist/swift-svn</font></div><div><font face="times new roman, new york, times, serif">/home/davidk/swift-faster-03012013/cog/modules/swift/dist/swift-svn </font></div><div><font face="times new roman, new york, times, serif">/home/davidk/swift-trunk-03012013/cog/modules/swift/dist/swift-svn</font><br><br>It's probably better to use these rather than ~davidk/swift-trunk, because I tend to use that for testing. Hope this helps.</div><div><br></div><div>David<br><br><hr id="zwchr" style="color: rgb(0, 0, 0); font-family: 'times new roman', 'new york', times, serif; font-size: 12pt;"><blockquote style="color: rgb(0, 0, 0); font-family: Helvetica, Arial, sans-serif; font-size: 12pt; border-left-width: 2px; border-left-style: solid; border-left-color: rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; font-weight: normal; font-style: normal; text-decoration: none;"><b>From: </b>"Lorenzo Pesce" <lpesce@uchicago.edu><br><b>To: </b>"David Kelly" <davidk@ci.uchicago.edu><br><b>Cc: </b>"Mihael Hategan" <hategan@mcs.anl.gov>, "Swift Devel" <swift-devel@ci.uchicago.edu><br><b>Sent: </b>Friday, March 1, 2013 3:32:20 PM<br><b>Subject: </b>Re: [Swift-devel] 0.94 release note draft<br><br>David,<div><br></div><div>Have mercy on me having lost track of the situation. Maybe tomorrow, most definitely next week I will start to increase the size of some of my swift runs till I hit the thousands of jobs.</div><div>Some will be very heavy in I/O and computations (and pipeline complexity).</div><div><br></div><div>I have taken today off to recover and be ready.</div><div><br></div><div>What do you suggest should be my approach to this problem? </div><div>I am currently trying to develop pipelines with </div><div><div><br></div><div>SWIFT_HOME=/soft/swift/0.94-2012.1102</div><div><br></div><div>or </div><div><br></div><div>#SWIFT_HOME=/home/davidk/swift-trunk/cog/modules/swift/dist/swift-svn</div></div><div><br></div><div>then move them to fast and see where it breaks.</div><div><br></div><div>Should i change the module I am using? I need to learn how to make the installation work and get over my hatred for java...</div><div><br></div><div>All the pipelines I will be trying are expected eventually to reach sustained runs of 10K+ jobs in parallel with workflows as deep as 20 stages (including steps that increase the jobs by one or two orders of magnitude with a total file load of a few tens of TBs)</div><div><br></div><div>Do you think that it is sensible?</div><div><br></div><div>We will then mover to other machines and hopefully test fusion and other approaches.</div><div><br></div><div>Lorenzo</div><div><br></div><div><br></div><div><div>On Mar 1, 2013, at 3:13 PM, David Kelly wrote:</div><br class="Apple-interchange-newline"><blockquote><div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; color: rgb(0, 0, 0); ">I think it is there. It looks like your memory fixes were committed on 2-2, and the current 0.94 branch is based on a snapshot of trunk from 2-15. <br><br><hr id="zwchr"><blockquote style="border-left-width: 2px; border-left-style: solid; border-left-color: rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; color: rgb(0, 0, 0); font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-size: 12pt; "><b>From:<span class="Apple-converted-space"> </span></b>"Mihael Hategan" <<a href="mailto:hategan@mcs.anl.gov" target="_blank">hategan@mcs.anl.gov</a>><br><b>To:<span class="Apple-converted-space"> </span></b>"David Kelly" <<a href="mailto:davidk@ci.uchicago.edu" target="_blank">davidk@ci.uchicago.edu</a>><br><b>Cc:<span class="Apple-converted-space"> </span></b>"Michael Wilde" <<a href="mailto:wilde@mcs.anl.gov" target="_blank">wilde@mcs.anl.gov</a>>, "Swift Devel" <<a href="mailto:swift-devel@ci.uchicago.edu" target="_blank">swift-devel@ci.uchicago.edu</a>><br><b>Sent:<span class="Apple-converted-space"> </span></b>Friday, March 1, 2013 12:36:03 PM<br><b>Subject:<span class="Apple-converted-space"> </span></b>Re: [Swift-devel] 0.94 release note draft<br><br>So I don't think I committed the memory leak fixes to 0.94 and I think<br>they should be there.<br><br>Mihael<br><br>On Fri, 2013-03-01 at 07:06 -0600, David Kelly wrote:<br>> Here is the list, based mostly on what I could find in the svn logs.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - To behavior of iterate has changed from 0.93 to 0.94. If you have scripts<span class="Apple-converted-space"> </span><br>> that use iterate, please read<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_iterate" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_iterate</a><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Walltimes are more strictly enforced by coasters. Previous to Swift version<span class="Apple-converted-space"> </span><br>> 0.94, if an application run with coasters would exceed its specified<span class="Apple-converted-space"> </span><br>> maxwalltime, it would be allowed to continue to execute. However, if this<span class="Apple-converted-space"> </span><br>> would cause the worker on which the application was running to exceed its<span class="Apple-converted-space"> </span><br>> maxwalltime, the queuing system would kill the worker. The resulting error<span class="Apple-converted-space"> </span><br>> message was not always very clear. Since version 0.94 coaster workers enforce<span class="Apple-converted-space"> </span><br>> the user-specified maxwalltime. If an application exceeds its maxwalltime,<span class="Apple-converted-space"> </span><br>> the coaster worker will not allow it to continue, but terminate it and report<span class="Apple-converted-space"> </span><br>> the error.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Swift will now use camel case for functions, for example, @toInt instead of<span class="Apple-converted-space"> </span><br>> @toint. The previous naming convention will still work, but you may see<span class="Apple-converted-space"> </span><br>> deprecated warnings.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Associative arrays have been added. More details and examples can be found at<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_associative_arrays" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_associative_arrays</a><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Dynamic profiles. Many settings formerly only definable in sites.xml can now<span class="Apple-converted-space"> </span><br>> be set on a per-app basis. This can make things easier when running<span class="Apple-converted-space"> </span><br>> multiple apps that have different requirements for settings like processors<span class="Apple-converted-space"> </span><br>> per node and wallitme.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_dynamic_profiles" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_dynamic_profiles</a><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Added a new ssh command line provider. Previously ssh support was done by<span class="Apple-converted-space"> </span><br>> creating a file called ~/.ssh/auth.defaults. The ssh command line provider<span class="Apple-converted-space"> </span><br>> is more flexible and doesn't require this step. ssh-cl allows you to use<span class="Apple-converted-space"> </span><br>> SSH agents. You can use ssh-cl by adding something like this to your sites.xml:<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> <execution provider="coaster" url="<a href="http://my.host.uchicago.edu" target="_blank">my.host.uchicago.edu</a>" jobmanager="ssh-cl:pbs"/><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Many fixes and improvements to improve the reliability and performance of<span class="Apple-converted-space"> </span><br>> coaster provider staging.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Added support for the Slurm scheduler<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Added support for the LSF scheduler<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Improvements to condor provider (non-shared jobtype and more flexibility<span class="Apple-converted-space"> </span><br>> to define what gets added to the submit script).<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Fixes for the textual user interface (TUI). Adding the -tui option to the swift<span class="Apple-converted-space"> </span><br>> command line allows you to monitor progress in a curses based menu. A brief<span class="Apple-converted-space"> </span><br>> example of this can be found at<span class="Apple-converted-space"> </span><a href="http://www.ci.uchicago.edu/~davidk/modis.ogv" target="_blank">http://www.ci.uchicago.edu/~davidk/modis.ogv</a>.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Added the ability to call Java methods within swift using @java. For example:<span class="Apple-converted-space"> </span><br>> float f = @java("java.lang.Math", "sin", 0.5);<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_java" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_java</a><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Added a hang checker that provides the user with more information about<span class="Apple-converted-space"> </span><br>> potential hangs<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - @strjoin function for joining strings.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><a href="http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_strjoin" target="_blank">http://www.ci.uchicago.edu/swift/guides/release-0.94/userguide/userguide.html#_strjoin</a><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - If you have a requirement that a command get run on the worker node before<span class="Apple-converted-space"> </span><br>> processing any work,<span class="Apple-converted-space"> </span><a href="http://worker.pl" target="_blank">worker.pl</a><span class="Apple-converted-space"> </span>will now execute commands stored in the<span class="Apple-converted-space"> </span><br>> environment variable $WORKER_INIT_CMD<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Use $SWIFT_USERHOME to determine where to swift should create some of its<span class="Apple-converted-space"> </span><br>> required files. This defaults to $HOME, but this may cause problems in some<span class="Apple-converted-space"> </span><br>> situations where $HOME is not accessable on worker nodes.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Experimental "wrapper staging" feature that delegates file staging to an<span class="Apple-converted-space"> </span><br>> external wrapper script.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Various improvements to the way that Swift runs MPI jobs.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Better OSG integration/support using GlideinWMS.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> ----- Original Message -----<br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> From: "Michael Wilde" <<a href="mailto:wilde@mcs.anl.gov" target="_blank">wilde@mcs.anl.gov</a>><span class="Apple-converted-space"> </span><br>> To: "David Kelly" <<a href="mailto:davidk@ci.uchicago.edu" target="_blank">davidk@ci.uchicago.edu</a>><span class="Apple-converted-space"> </span><br>> Cc: "Swift Devel" <<a href="mailto:swift-devel@ci.uchicago.edu" target="_blank">swift-devel@ci.uchicago.edu</a>><span class="Apple-converted-space"> </span><br>> Sent: Thursday, February 14, 2013 1:08:56 PM<span class="Apple-converted-space"> </span><br>> Subject: Fwd: 0.94 release note draft<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> Some notes toward an 0.94 release notes document.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> There's a longer list, I think in an IM chat transcript, that we need to incorporate.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> Please send additional items to this thread for David to integrate.<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> Thanks,<span class="Apple-converted-space"> </span><br>><span class="Apple-converted-space"> </span><br>> - Mike<span class="Apple-converted-space"> </span><br><br><br></blockquote><br></div>_______________________________________________<br>Swift-devel mailing list<br><a href="mailto:Swift-devel@ci.uchicago.edu" target="_blank">Swift-devel@ci.uchicago.edu</a><br><a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br></div></blockquote></div><br></blockquote><br></div></div></body></html>