From noreply at svn.ci.uchicago.edu  Fri Oct  3 08:39:40 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 08:39:40 -0500 (CDT)
Subject: [Swift-commit] r2250 - trunk/libexec
Message-ID: <20081003133940.1E0371360037@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-03 08:39:38 -0500 (Fri, 03 Oct 2008)
New Revision: 2250

Modified:
   trunk/libexec/vdl-int.k
Log:
stageout destination directory needs to be created using same destination host and provider as the actual stageout.

Modified: trunk/libexec/vdl-int.k
===================================================================
--- trunk/libexec/vdl-int.k	2008-09-23 19:29:16 UTC (rev 2249)
+++ trunk/libexec/vdl-int.k	2008-10-03 13:39:38 UTC (rev 2250)
@@ -241,7 +241,7 @@
 					log(LOG:DEBUG, "FILE_STAGE_OUT_START srcname={bname} srcdir={rdir} srchost={host} ",
 							"destdir={ldir} desthost={dhost} provider={provider}")
 					//make sure we do have the directory on the client side
-					dir:make(ldir)
+					dir:make(ldir, host=dhost, provider=provider)
 					restartOnError(".*", 2
 					    task:transfer(srchost=host, srcfile=bname,
 					        srcdir=rdir, destdir=ldir, desthost=dhost, destprovider=provider)



From noreply at svn.ci.uchicago.edu  Fri Oct  3 08:44:52 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 08:44:52 -0500 (CDT)
Subject: [Swift-commit] r2251 - nmi-build-test
Message-ID: <20081003134452.32DE41360037@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-03 08:44:51 -0500 (Fri, 03 Oct 2008)
New Revision: 2251

Modified:
   nmi-build-test/coaster
Log:
coaster test does not work (mostly because of missing homedir, I think) on NMI; but we might as well still try building it to catch compile errors.

Modified: nmi-build-test/coaster
===================================================================
--- nmi-build-test/coaster	2008-10-03 13:39:38 UTC (rev 2250)
+++ nmi-build-test/coaster	2008-10-03 13:44:51 UTC (rev 2251)
@@ -8,9 +8,9 @@
 echo building...
 ant redist -Dwith-provider-coaster=true || exit 1
 
-echo testing...
-export PATH=`pwd`/dist/vdsk-svn/bin:$PATH
+# echo testing...
+# export PATH=`pwd`/dist/vdsk-svn/bin:$PATH
 
-cd tests/misc
-./coaster.sh || exit 4
+# cd tests/misc
+# ./coaster.sh || exit 4
 



From noreply at svn.ci.uchicago.edu  Fri Oct  3 12:34:35 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 12:34:35 -0500 (CDT)
Subject: [Swift-commit] r2253 - log-processing
Message-ID: <20081003173435.EEA271360037@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-03 12:34:34 -0500 (Fri, 03 Oct 2008)
New Revision: 2253

Modified:
   log-processing/README
Log:
tidy up basic usage section to reflect presence of command line utility to make a plot

Modified: log-processing/README
===================================================================
--- log-processing/README	2008-10-03 14:47:33 UTC (rev 2252)
+++ log-processing/README	2008-10-03 17:34:34 UTC (rev 2253)
@@ -1,3 +1,4 @@
+This is the README for the swift log-processing package.
 
 Some prerequisites
 ==================
@@ -9,30 +10,27 @@
 Basic use
 =========
 
-To generate webpage:
+To generate a webpage:
 
-Run like this:
+add <CHECKOUT>/bin to your path
 
-make LOG=raw.log clean webpage
+swift-plot-log modelproc-20080918-1644-nyjqiddf.log
 
-you can specify any swift log file
+You will be left with a directory
 
-you'll get an index.html with some embedded png links
+report-modelproc-20080918-1644-nyjqiddf/
 
-Kickstart
-=========
+in which you will find
 
-If you want to process kickstart records, specify an additional
-parameter KDIR=/path/to/where/kickstart/records/are
+report-modelproc-20080918-1644-nyjqiddf/index.html
 
-Falkon
-======
-If you want to process falkon logs, specify FDIR=/path/to/falkon/log/dir
-and add webpage.falkon as a makefile target
+Open that with your favourite web browser.
 
-Commandline Tools
-=================
 
+
+Other Commandline Tools
+=======================
+
   ./execute2-status-from-log [logfile]
      lists every (execute2) job and its final status
 



From noreply at svn.ci.uchicago.edu  Fri Oct  3 14:51:20 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 14:51:20 -0500 (CDT)
Subject: [Swift-commit] r2254 - adem
Message-ID: <20081003195120.D3DCE1360037@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 14:51:19 -0500 (Fri, 03 Oct 2008)
New Revision: 2254

Added:
   adem/nab-5.1.2.tar.gz
Log:
Nab import

Added: adem/nab-5.1.2.tar.gz
===================================================================
(Binary files differ)


Property changes on: adem/nab-5.1.2.tar.gz
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream



From noreply at svn.ci.uchicago.edu  Fri Oct  3 15:34:19 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 15:34:19 -0500 (CDT)
Subject: [Swift-commit] r2256 - in adem: . nab
Message-ID: <20081003203419.1FBD91360037@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 15:34:17 -0500 (Fri, 03 Oct 2008)
New Revision: 2256

Added:
   adem/nab/
   adem/nab/nab-5.1.2.tar.gz
Log:
Nab import

Added: adem/nab/nab-5.1.2.tar.gz
===================================================================
(Binary files differ)


Property changes on: adem/nab/nab-5.1.2.tar.gz
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream



From noreply at svn.ci.uchicago.edu  Fri Oct  3 15:44:37 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 15:44:37 -0500 (CDT)
Subject: [Swift-commit] r2257 - /
Message-ID: <20081003204437.28C721360037@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 15:44:36 -0500 (Fri, 03 Oct 2008)
New Revision: 2257

Added:
   nab/
Log:
Making nab dir



From noreply at svn.ci.uchicago.edu  Fri Oct  3 15:47:11 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 15:47:11 -0500 (CDT)
Subject: [Swift-commit] r2258 - /
Message-ID: <20081003204711.C30091360037@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 15:47:11 -0500 (Fri, 03 Oct 2008)
New Revision: 2258

Removed:
   nab/
Log:
Delete nab dir



From noreply at svn.ci.uchicago.edu  Fri Oct  3 15:52:04 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 15:52:04 -0500 (CDT)
Subject: [Swift-commit] r2259 - /
Message-ID: <20081003205204.AFC1E1360037@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 15:52:03 -0500 (Fri, 03 Oct 2008)
New Revision: 2259

Added:
   nab/
Log:
Making nab dir



From noreply at svn.ci.uchicago.edu  Fri Oct  3 15:53:25 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 15:53:25 -0500 (CDT)
Subject: [Swift-commit] r2260 - nab
Message-ID: <20081003205325.4C0EA1360037@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 15:53:23 -0500 (Fri, 03 Oct 2008)
New Revision: 2260

Added:
   nab/nab-5.1.2.tar.gz
Log:
 nab 

Added: nab/nab-5.1.2.tar.gz
===================================================================
(Binary files differ)


Property changes on: nab/nab-5.1.2.tar.gz
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream



From noreply at svn.ci.uchicago.edu  Fri Oct  3 15:54:21 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 15:54:21 -0500 (CDT)
Subject: [Swift-commit] r2261 - /
Message-ID: <20081003205421.7C6C41360037@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 15:54:21 -0500 (Fri, 03 Oct 2008)
New Revision: 2261

Removed:
   nab/
Log:
nab dir



From noreply at svn.ci.uchicago.edu  Fri Oct  3 15:54:43 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 15:54:43 -0500 (CDT)
Subject: [Swift-commit] r2262 - /
Message-ID: <20081003205443.85BB61360037@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 15:54:43 -0500 (Fri, 03 Oct 2008)
New Revision: 2262

Added:
   nab-5.1.2/
Log:
Making nab dir



From noreply at svn.ci.uchicago.edu  Fri Oct  3 16:51:50 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 16:51:50 -0500 (CDT)
Subject: [Swift-commit] r2264 - /
Message-ID: <20081003215150.D1D8D9B4001@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 16:51:49 -0500 (Fri, 03 Oct 2008)
New Revision: 2264

Removed:
   nab/
Log:
delete nab



From noreply at svn.ci.uchicago.edu  Fri Oct  3 16:52:26 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 16:52:26 -0500 (CDT)
Subject: [Swift-commit] r2265 - /
Message-ID: <20081003215226.04AD99B4001@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 16:52:25 -0500 (Fri, 03 Oct 2008)
New Revision: 2265

Removed:
   nab-5.1.2/
Log:
delete nab



From noreply at svn.ci.uchicago.edu  Fri Oct  3 16:58:46 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  3 Oct 2008 16:58:46 -0500 (CDT)
Subject: [Swift-commit] r2266 - adem
Message-ID: <20081003215846.981B19B4001@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-03 16:58:45 -0500 (Fri, 03 Oct 2008)
New Revision: 2266

Removed:
   adem/nab/
Log:
nab



From noreply at svn.ci.uchicago.edu  Sat Oct  4 05:01:36 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sat,  4 Oct 2008 05:01:36 -0500 (CDT)
Subject: [Swift-commit] r2267 - trunk/docs
Message-ID: <20081004100136.D4A789B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-04 05:01:35 -0500 (Sat, 04 Oct 2008)
New Revision: 2267

Modified:
   trunk/docs/quickstartguide.xml
   trunk/docs/userguide.xml
Log:
remove most references to $SWIFT_HOME as it was incorrectly suggestive that such variable should habitually be set.

Modified: trunk/docs/quickstartguide.xml
===================================================================
--- trunk/docs/quickstartguide.xml	2008-10-03 21:58:45 UTC (rev 2266)
+++ trunk/docs/quickstartguide.xml	2008-10-04 10:01:35 UTC (rev 2267)
@@ -78,10 +78,8 @@
 				build instructions are available on the <ulink
 				url="http://www.ci.uchicago.edu/swift/downloads/index.php#nightly">Swift
 				downloads page</ulink>. Once built, the <filename
-				class="directory">dist/vdsk-&lt;version&gt;</filename> directory
-				will contain a self-contained build which can be moved to a
-				desired location. For simplicity this directory will be called
-				<filename class="directory"><envar>$SWIFT_HOME</envar></filename>.
+				class="directory">dist/vdsk-svn</filename> directory
+				will contain a self-contained build which can be used in place or moved to a different location.
 				You should then proceed to the <link
 				linkend="configure">configuration section</link>.
 			
@@ -105,9 +103,7 @@
 			
 			This will create a <filename
 			class="directory">vdsk-&lt;version&gt;</filename> directory
-			containing the build. For simplicity this directory will be refered
-			to as <filename
-			class="directory"><envar>$SWIFT_HOME</envar></filename>.
+			containing the build.
 		
 		</para>
 	</sect1>
@@ -120,13 +116,13 @@
 			This section describes configuration steps that need to be taken in
 			order to get Swift running. Since all command line tools provided
 			with Swift can be found in the <filename
-			class="directory"><envar>$SWIFT_HOME</envar>/bin</filename> directory, it may
+			class="directory">bin/</filename> directory of the Swift distribution, it may
 			be a good idea to add this directory to your <envar>PATH</envar>
 			environment variable:
 			
 <programlisting>
 <prompt>&gt;</prompt> <command>export</command> <envar>PATH</envar>=<filename
-class="directory"><envar>$SWIFT_HOME</envar>/bin</filename>:<envar>$PATH</envar>
+class="directory">/path/to/swift/bin</filename>:<envar>$PATH</envar>
 </programlisting>
 			
 			If you are installing Swift on a machine with access to a graphical
@@ -202,7 +198,8 @@
 				class="file">swift.properties</filename>) can be used to
 				customize certain configuration aspects of Swift. A shared
 				version of this file, <filename
-				class="file"><envar>$SWIFT_HOME</envar>/etc/swift.properties</filename>
+				class="file">etc/swift.properties</filename>
+				in the installation directory
 				can be used to provide installation-wide defaults. A per-user
 				properties file, <filename
 				class="file">~/.swift/swift.properties</filename> can be used for
@@ -235,7 +232,7 @@
 		<para>
 		
 			The Swift examples can be found in the <filename
-			class="directory">&lt;SWIFT_HOME&gt;/examples</filename> directory.
+			class="directory">examples</filename> directory in the Swift distribution.
 			The examples are written in the <ulink
 			url="http://www.ci.uchicago.edu/swift/guides/languagespec.php">SwiftScript
 			Language</ulink>, and have <filename class="file">.swift</filename> as

Modified: trunk/docs/userguide.xml
===================================================================
--- trunk/docs/userguide.xml	2008-10-03 21:58:45 UTC (rev 2266)
+++ trunk/docs/userguide.xml	2008-10-04 10:01:35 UTC (rev 2267)
@@ -1009,7 +1009,7 @@
 			Various aspects of the behavior of the Swift Engine can be
 			configured through properties. The Swift Engine recognizes a global,
 			per installation properties file which can found in <filename
-			class="file">$SWIFT_HOME/etc/swift.properties</filename> and a user
+			class="file">etc/swift.properties</filename> in the Swift installation directory and a user
 			properties file which can be created by each user in <filename
 			class="file">~/.swift/swift.properties</filename>. The Swift Engine
 			will first load the global properties file. It will then try to load
@@ -1048,7 +1048,9 @@
 							
 								Points to the Swift installation directory
 								(<filename
-								class="directory"><envar>$SWIFT_HOME</envar></filename>).
+								class="directory"><envar>$SWIFT_HOME</envar></filename>). In general, this should not be set
+as Swift can find its own installation directory, and incorrectly setting it
+may impair the correct functionality of Swift.
 							
 							</para>
 						</listitem>



From noreply at svn.ci.uchicago.edu  Sat Oct  4 20:49:52 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sat,  4 Oct 2008 20:49:52 -0500 (CDT)
Subject: [Swift-commit] r2268 - /
Message-ID: <20081005014952.E611F22819E@www.ci.uchicago.edu>

Author: leggett
Date: 2008-10-04 20:49:51 -0500 (Sat, 04 Oct 2008)
New Revision: 2268

Added:
   ti-test/
Log:
- testing a commit




From noreply at svn.ci.uchicago.edu  Sat Oct  4 20:51:54 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sat,  4 Oct 2008 20:51:54 -0500 (CDT)
Subject: [Swift-commit] r2269 - /
Message-ID: <20081005015154.BE0EF228192@www.ci.uchicago.edu>

Author: leggett
Date: 2008-10-04 20:51:54 -0500 (Sat, 04 Oct 2008)
New Revision: 2269

Removed:
   ti-test/
Log:
- removing test commit




From noreply at svn.ci.uchicago.edu  Sat Oct  4 21:02:53 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sat,  4 Oct 2008 21:02:53 -0500 (CDT)
Subject: [Swift-commit] r2270 - log-processing/libexec
Message-ID: <20081005020253.4CBA6228192@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-04 21:02:52 -0500 (Sat, 04 Oct 2008)
New Revision: 2270

Added:
   log-processing/libexec/assorted.html.template
   log-processing/libexec/execute.html.template
   log-processing/libexec/execute2.html.template
   log-processing/libexec/falkon.html.template
   log-processing/libexec/info.html.template
   log-processing/libexec/karajan.html.template
   log-processing/libexec/kickstart.html.template
   log-processing/libexec/scheduler.html.template
   log-processing/libexec/toolbar.html.template
Modified:
   log-processing/libexec/index.html.template
   log-processing/libexec/log-to-karatasks-transitions
   log-processing/libexec/makefile
   log-processing/libexec/makefile.webpage
Log:
rearrange generated web page into a multipage format, hopefully to make it easier to navigate and provide more structure into which to put fascinating descriptions of what each graph is about...

Added: log-processing/libexec/assorted.html.template
===================================================================
--- log-processing/libexec/assorted.html.template	                        (rev 0)
+++ log-processing/libexec/assorted.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -0,0 +1,38 @@
+<html>
+<head><title>Swift log analysis</title></head>
+
+<body><h1>Swift log analysis</h1>
+
+include(`toolbar.html.template')
+
+<h2>Assorted Swift library routines</h2>
+
+<p>Shared directory initialisation on remote sites:</p>
+<img src="initshareddir.png"/>
+
+<p>Per-job shared directory creation on remote sites:</p>
+<img src="createdirset.png"/>
+<br/>
+<img src="createdirset-total.png"/>
+
+<p>dostagein:</p>
+<img src="dostagein.png"/>
+<br/>
+<img src="dostagein-total.png"/>
+<p>dostagein duration histogram:</p>
+<img src="dostagein-duration-histogram.png" />
+<hr />
+
+<p>dostageout:</p>
+<img src="dostageout.png"/>
+<br/>
+<img src="dostageout-total.png"/>
+<p>dostageout sorted by start time</p>
+<img src="dostageout.sorted-start.png"/>
+
+<p>dostageout duration histogram:</p>
+<img src="dostageout-duration-histogram.png" />
+<hr />
+
+<p>end</p>
+</body>

Added: log-processing/libexec/execute.html.template
===================================================================
--- log-processing/libexec/execute.html.template	                        (rev 0)
+++ log-processing/libexec/execute.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -0,0 +1,55 @@
+<html>
+<head><title>Swift log analysis</title></head>
+
+<body><h1>Swift log analysis</h1>
+
+include(`toolbar.html.template')
+
+<!-- (TODO indicate log file name here; indicate report generation
+time here; indicate version of swift that was used - if locally modified
+print a big red warning) -->
+
+<hr />
+<h2>Summary of execution</h2>
+<p>This plot shows the state of executions over time. When a SwiftScript
+program makes a procedure call, an execution will appear on this plot
+in START state. Over time, jobs should end up either in END_SUCCESS or
+END_FAILURE states.
+</p>
+<img src="execute-trails.png" />
+
+<p>Final status totals:</p>
+<pre>
+include(`execute.lastsummary')
+</pre>
+
+<h2>execute</h2>
+<p>Executions 
+(<a href="execute.event">text file of events</a>)
+(these are what appear as procedure invocations in SwiftScript
+and are rougly analogous to VDL1 derivations):</p>
+<img src="execute.png" />
+<p>The number of execute tasks in progress at once (for each time t, count the 
+number of event lines on the above graph that are drawn at time t):</p>
+<img src="execute-total.png"/>
+
+<p>Execute events ordered by start time</p>
+<img src="execute.sorted-start.png" />
+
+<p>Breakdown of last known status for execute:</p>
+<pre>
+include(`execute.lastsummary')
+</pre>
+
+<p>execute trails:</p>
+<img src="execute-trails.png" />
+
+<hr />
+<h2>general stats</h2>
+<p>Some execution counts:</p>
+<pre>
+include(`execution-counts.txt')
+</pre>
+
+<p>end</p>
+</body>

Added: log-processing/libexec/execute2.html.template
===================================================================
--- log-processing/libexec/execute2.html.template	                        (rev 0)
+++ log-processing/libexec/execute2.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -0,0 +1,56 @@
+<html>
+<head><title>Swift log analysis</title></head>
+
+<body><h1>Swift log analysis</h1>
+
+include(`toolbar.html.template')
+
+<hr />
+
+<p>Breakdown of execute2 final statuses by site:
+(<a href="execute2.html">More information about execute2 events</a>)</p>
+
+include(`jobs-sites.table')
+
+<p>Number of execute2 jobs at once - this is basically the number of
+execution attempts that have been submitted to some batch system such
+as GRAM (so are either queued or executing)</p>
+<img src="execute2-total.png"/>
+
+<h2>execute2</h2>
+<p>execute2 tasks (<a href="execute2.event">text file of events</a>) - these are executions of programs. Sometimes (in the
+case of failure) there will be multiple execute2 tasks per execute task:</p>
+<img src="execute2.png" />
+<p>number of execute2 tasks at once:</p>
+<img src="execute2-total.png"/>
+<p>execute2 tasks, coloured by site</p>
+<img src="execute2-coloured.png" />
+<p>Breakdown of last known status for execute2s:</p>
+<pre>
+include(`execute2.lastsummary')
+</pre>
+<p>execute2 trails:</p>
+
+<img src="execute2-trails.png" />
+
+<p>Summary of execute2 transformation names that have been seen
+(eg are enqueued or have been run):</p>
+<pre>
+include(`trname-summary')
+</pre>
+
+<p>per-site execute2 duration statistics</p>
+<pre>
+include(`site-duration.txt')
+</pre>
+
+<p>Summary of execute2s/execute:
+</p>
+<pre>
+|number of executes | number of execute2s in each execute |
+include(`jobs.retrycount.summary')
+</pre>
+
+<hr />
+<p>end</p>
+</body>

Added: log-processing/libexec/falkon.html.template
===================================================================
--- log-processing/libexec/falkon.html.template	                        (rev 0)
+++ log-processing/libexec/falkon.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -0,0 +1,20 @@
+<html>
+<head><title>Swift log analysis</title></head>
+
+<body><h1>Swift log analysis</h1>
+
+include(`toolbar.html.template')
+
+<h2>falkon</h2>
+<p>Falkon taskperf log:</p>
+
+<img src="falkon.png"/>
+
+<p>Total jobs that falkon has queued or running:</p>
+<img src="falkon-total.png"/>
+
+<p>Falkon job status trails:</p>
+<img src="falkon-task-status-trails.png"/>
+
+<p>end</p>
+</body>

Modified: log-processing/libexec/index.html.template
===================================================================
--- log-processing/libexec/index.html.template	2008-10-05 01:51:54 UTC (rev 2269)
+++ log-processing/libexec/index.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -3,251 +3,32 @@
 
 <body><h1>Swift log analysis</h1>
 
-<p>(TODO indicate log file name here; indicate report generation
+include(`toolbar.html.template')
+
+<!-- (TODO indicate log file name here; indicate report generation
 time here; indicate version of swift that was used - if locally modified
-print a big red warning)</p>
+print a big red warning) -->
 
-<hr />
 <h2>Summary of execution</h2>
 <p>This plot shows the state of executions over time. When a SwiftScript
 program makes a procedure call, an execution will appear on this plot
 in START state. Over time, jobs should end up either in END_SUCCESS or
-END_FAILURE states. (<a href="execute.html">More information about
+END_FAILURE states. (<a href="execute.html">More detail about
 execute events</a>)
 </p>
 <img src="execute-trails.png" />
 
-<p>Final status totals:</p>
-<pre>
-include(`execute.lastsummary')
-</pre>
-
-<p>Breakdown of execute2 final statuses by site:
-(<a href="execute2.html">More information about execute2 events</a>)</p>
-
-include(`jobs-sites.table')
-
-
-<p>Number of execute2 jobs at once - this is basically the number of
-execution attempts that have been submitted to some batch system such
-as GRAM (so are either queued or executing)</p>
-<img src="execute2-total.png"/>
-
 <p>Number of karajan level job submissions that are 'Active' - this means
 that the underlying execution layer (such as GRAM) has reported that the
-job is actually executing</p>
+job is actually executing - this is an approximation of the number
+of cores in use at once (for single threaded application code)
+(<a href="karajan.html">More detail about karajan events</a>)</p>
 <img src="karatasks.JOB_SUBMISSION.Active-total.png" />
 
-<hr />
-<h2>execute</h2>
-<p>Executions 
-(<a href="execute.event">text file of events</a>)
-(these are what appear as procedure invocations in SwiftScript
-and are rougly analogous to VDL1 derivations):</p>
-<img src="execute.png" />
-<p>The number of execute tasks in progress at once (for each time t, count the 
-number of event lines on the above graph that are drawn at time t):</p>
-<img src="execute-total.png"/>
-
-<p>Execute events ordered by start time</p>
-<img src="execute.sorted-start.png" />
-
-<p>Breakdown of last known status for execute:</p>
-<pre>
-include(`execute.lastsummary')
-</pre>
-
-<p>execute trails:</p>
-<img src="execute-trails.png" />
-
-<hr />
-<h2>execute2</h2>
-<p>execute2 tasks (<a href="execute2.event">text file of events</a>) - these are executions of programs. Sometimes (in the
-case of failure) there will be multiple execute2 tasks per execute task:</p>
-<img src="execute2.png" />
-<p>number of execute2 tasks at once:</p>
-<img src="execute2-total.png"/>
-<p>execute2 tasks, coloured by site</p>
-<img src="execute2-coloured.png" />
-<p>Breakdown of last known status for execute2s:</p>
-<pre>
-include(`execute2.lastsummary')
-</pre>
-<p>execute2 trails:</p>
-
-<img src="execute2-trails.png" />
-
-<p>Summary of execute2 transformation names that have been seen
-(eg are enqueued or have been run):</p>
-<pre>
-include(`trname-summary')
-</pre>
-
-
-<hr />
-<h2>kickstart</h2>
-<p>kickstart records.
-(<a href="kickstart.event">text file of events</a>)
-this shows applications which are actually running on a worker node, executing:</p>
-<img src="kickstart.png" />
-
-<p>kickstart records total - this represents how many worker nodes were
-actually executing applications at any time:</p>
-<img src="kickstart-total.png" />
-
-<p>kickstart records sorted by start time:</p>
-<img src="kickstart.sorted-start.png" />
-
-<p>kickstart stats:
-<pre>
-include(`kickstart.stats')
-</pre>
-</p>
-
-<p>cumulative histogram(?) of kickstart job durations</p>
-
-<img src="kickstart-duration-histogram.png" />
-
-<hr />
-<h2>karajan tasks</h2>
-<p>just karajan task start/end
-(<a href="karatasks.event">text file of events</a>):</p>
-<img src="karatasks.png" />
-
-<p>karajan tasks, coloured by type:</p>
-<img src="karatasks-coloured.png" />
-
-<p>karajan tasks, coloured by type and sorted by start time:</p>
-<img src="karatasks-coloured-sorted.png" />
-
-<p>Total karajan tasks in progress at any time:</p>
-<img src="karatasks-total.png" />
-
-<p>Counts for each task type:</p>
-
-<pre>
-include(`karatasks-type-counts.txt')
-</pre>
-
-<p>Breakdown of last known status for karajan tasks:</p>
-<pre>
-include(`karatasks.lastsummary')
-</pre>
-
-<p>karajan FILE_TRANSFER tasks</p>
-<img src="karatasks.FILE_TRANSFER.sorted-start.png" />
-<br/>
-<img src="karatasks.FILE_TRANSFER-total.png" />
-<br/>
-<pre>
-include(`karatasks.FILE_TRANSFER.stats')
-</pre>
-
-<p>karatasks FILE_TRANSFER trails:</p>
-<img src="karatasks.FILE_TRANSFER-trails.png" />
-
-<p>karajan FILE_OPERATION tasks</p>
-<img src="karatasks.FILE_OPERATION.sorted-start.png" />
-<br/>
-<img src="karatasks.FILE_OPERATION-total.png" />
-<br/>
-<pre>
-include(`karatasks.FILE_OPERATION.stats')
-</pre>
-<p>karatasks FILE_OPERATION trails:</p>
-<img src="karatasks.FILE_OPERATION-trails.png" />
-<p>karajan JOB_SUBMISSION tasks</p>
-<img src="karatasks.JOB_SUBMISSION.sorted-start.png" />
-<br/>
-<img src="karatasks.JOB_SUBMISSION-total.png" />
-<br/>
-<pre>
-include(`karatasks.JOB_SUBMISSION.stats')
-</pre>
-<p>karatasks JOB_SUBMISSION trails:</p>
-<img src="karatasks.JOB_SUBMISSION-trails.png" />
-
-<p>Number of karajan level job submissions that are 'Submitted' - this means
-that the underlying execution layer (such as GRAM) has reported that the
-job is submitted for execution, but not executing (queued).</p>
-<img src="karatasks.JOB_SUBMISSION.Queue-total.png" />
-
-<p>karajan queued JOB_SUBMISSION tasks</p>
-<img src="karatasks.JOB_SUBMISSION.Queue.sorted-start.png" />
-
-<p>karajan queued JOB_SUBMISSION cumulative duration</p>
-<img src="karatasks.JOB_SUBMISSION.Queue-duration-histogram.png" />
-
-<pre>
-include(`karatasks.JOB_SUBMISSION.Queue.stats')
-</pre>
-<hr />
-
-<p>Number of tasks in Active state</p>
-<img src="karatasks.JOB_SUBMISSION.Active-total.png" />
-
-<p>karajan active JOB_SUBMISSION tasks</p>
-<img src="karatasks.JOB_SUBMISSION.Active.sorted-start.png" />
-
-<p>karajan active JOB_SUBMISSION cumulative duration</p>
-<img src="karatasks.JOB_SUBMISSION.Active-duration-histogram.png" />
-
-<pre>
-include(`karatasks.JOB_SUBMISSION.Active.stats')
-</pre>
-<hr />
-
-<hr />
-<h2>falkon</h2>
-<p>Falkon taskperf log:</p>
-
-<img src="falkon.png"/>
-
-<p>Total jobs that falkon has queued or running:</p>
-<img src="falkon-total.png"/>
-
-<p>Falkon job status trails:</p>
-<img src="falkon-task-status-trails.png"/>
-
-<h2>various Swift karajan-level routines</h2>
-
-<p>Shared directory initialisation on remote sites:</p>
-<img src="initshareddir.png"/>
-
-<p>Per-job shared directory creation on remote sites:</p>
-<img src="createdirset.png"/>
-<br/>
-<img src="createdirset-total.png"/>
-
-<p>dostagein:</p>
-<img src="dostagein.png"/>
-<br/>
-<img src="dostagein-total.png"/>
-<p>dostagein duration histogram:</p>
-<img src="dostagein-duration-histogram.png" />
-<hr />
-
-<p>dostageout:</p>
-<img src="dostageout.png"/>
-<br/>
-<img src="dostageout-total.png"/>
-<p>dostageout sorted by start time</p>
-<img src="dostageout.sorted-start.png"/>
-
-<p>dostageout duration histogram:</p>
-<img src="dostageout-duration-histogram.png" />
-<hr />
-
-
 <p>Various execution-related total-event plots on one chart:</p>
 <img src="execstages.png" />
 
 <hr />
-<h2>everything</h2>
-<p>all events together - this one is rather meaningless:</p>
-<img src="all.png" />
-
-<hr />
 <h2>general stats</h2>
 <p>Some execution counts:</p>
 <pre>
@@ -259,71 +40,9 @@
 include(`sites-list.data')
 </pre>
 
-<p>sites/success table (according to execute2 events 
-<a href="execute2.event">execute2.event</a>):</p>
-
-include(`jobs-sites.table')
-
-<p>per-site execute2 duration statistics</p>
-<pre>
-include(`site-duration.txt')
-</pre>
-
-<p>Summary of execute2s/execute:
-</p>
-<pre>
-|number of executes | number of execute2s in each execute |
-include(`jobs.retrycount.summary')
-</pre>
-
-<hr />
-<h2>scheduler scores</h2>
-<p>Scheduler host scores:</p>
-<img src="weights.png" />
-
-<p>Scheduler load information:</p>
-<img src="tscores.png" />
-
-<hr />
-<h2>worker.sh info files</h2>
-<p>info file plots:</p>
-<p>info files:</p>
-<img src="info.png" />
-<p>info files sorted by start time:</p>
-<img src="info.sorted-start.png" />
-<p>info trails:</p>
-<img src="info-trails.png" />
-<p>info duration histogram:</p>
-<img src="info-duration-histogram.png" />
-<p>how wrapper.sh is spending its time:</p>
-<img src="info.zeroed-trailsx.png" />
-
-<p>total concurrent wrapper+application execution on worker nodes</p>
-
-<img src="info-total.png" />
-
-<p>info/wrapperlog stats:
-<pre>
-include(`info.stats')
-</pre>
-</p>
-<hr />
-
-
-1. <img src="karatasks.clusters.png" />  
-<br />
-2. <img src="karatasks.clusters-total.png" /> 
-<br />
-3. <img src="karatasks.clusters.sorted-start.png" />
-<br />
-4. <img src="karatasks.clusters-duration-histogram.png" />
-<br />
-5. <img src="cluster-jobcount-histogram.png" />
-<hr />
-
 <h2>Workflow</h2>
 
-<p>The workflow itself
+<p>The whole workflow
 (<a href="workflow.event">text file of event(s)</a>)
  - there should be just one of them and it should
 take the whole span of execution time:</p>

Added: log-processing/libexec/info.html.template
===================================================================
--- log-processing/libexec/info.html.template	                        (rev 0)
+++ log-processing/libexec/info.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -0,0 +1,29 @@
+<html>
+<head><title>Swift log analysis</title></head>
+
+<body><h1>Swift log analysis</h1>
+
+include(`toolbar.html.template')
+
+<hr />
+<h2>worker.sh info files</h2>
+<p>info file plots:</p>
+<p>info files:</p>
+<img src="info.png" />
+<p>info files sorted by start time:</p>
+<img src="info.sorted-start.png" />
+<p>info trails:</p>
+<img src="info-trails.png" />
+<p>info duration histogram:</p>
+<img src="info-duration-histogram.png" />
+<p>how wrapper.sh is spending its time:</p>
+<img src="info.zeroed-trailsx.png" />
+
+<p>total concurrent wrapper+application execution on worker nodes</p>
+
+<img src="info-total.png" />
+
+<hr />
+<p>end</p>
+</body>
+</html>

Added: log-processing/libexec/karajan.html.template
===================================================================
--- log-processing/libexec/karajan.html.template	                        (rev 0)
+++ log-processing/libexec/karajan.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -0,0 +1,121 @@
+<html>
+<head><title>Swift log analysis</title></head>
+
+<body><h1>Swift log analysis</h1>
+
+include(`toolbar.html.template')
+
+<!-- (TODO indicate log file name here; indicate report generation
+time here; indicate version of swift that was used - if locally modified
+print a big red warning) -->
+
+<hr />
+<p>Number of karajan level job submissions that are 'Active' - this means
+that the underlying execution layer (such as GRAM) has reported that the
+job is actually executing</p>
+<img src="karatasks.JOB_SUBMISSION.Active-total.png" />
+
+<hr />
+<h2>karajan tasks</h2>
+<p>just karajan task start/end
+(<a href="karatasks.event">text file of events</a>):</p>
+<img src="karatasks.png" />
+
+<p>karajan tasks, coloured by type:</p>
+<img src="karatasks-coloured.png" />
+
+<p>karajan tasks, coloured by type and sorted by start time:</p>
+<img src="karatasks-coloured-sorted.png" />
+
+<p>Total karajan tasks in progress at any time:</p>
+<img src="karatasks-total.png" />
+
+<p>Counts for each task type:</p>
+
+<pre>
+include(`karatasks-type-counts.txt')
+</pre>
+
+<p>Breakdown of last known status for karajan tasks:</p>
+<pre>
+include(`karatasks.lastsummary')
+</pre>
+
+<p>karajan FILE_TRANSFER tasks</p>
+<img src="karatasks.FILE_TRANSFER.sorted-start.png" />
+<br/>
+<img src="karatasks.FILE_TRANSFER-total.png" />
+<br/>
+<pre>
+include(`karatasks.FILE_TRANSFER.stats')
+</pre>
+
+<p>karatasks FILE_TRANSFER trails:</p>
+<img src="karatasks.FILE_TRANSFER-trails.png" />
+
+<p>karajan FILE_OPERATION tasks</p>
+<img src="karatasks.FILE_OPERATION.sorted-start.png" />
+<br/>
+<img src="karatasks.FILE_OPERATION-total.png" />
+<br/>
+<pre>
+include(`karatasks.FILE_OPERATION.stats')
+</pre>
+<p>karatasks FILE_OPERATION trails:</p>
+<img src="karatasks.FILE_OPERATION-trails.png" />
+<p>karajan JOB_SUBMISSION tasks</p>
+<img src="karatasks.JOB_SUBMISSION.sorted-start.png" />
+<br/>
+<img src="karatasks.JOB_SUBMISSION-total.png" />
+<br/>
+<pre>
+include(`karatasks.JOB_SUBMISSION.stats')
+</pre>
+<p>karatasks JOB_SUBMISSION trails:</p>
+<img src="karatasks.JOB_SUBMISSION-trails.png" />
+
+<p>Number of karajan level job submissions that are 'Submitted' - this means
+that the underlying execution layer (such as GRAM) has reported that the
+job is submitted for execution, but not executing (queued).</p>
+<img src="karatasks.JOB_SUBMISSION.Queue-total.png" />
+
+<p>karajan queued JOB_SUBMISSION tasks</p>
+<img src="karatasks.JOB_SUBMISSION.Queue.sorted-start.png" />
+
+<p>karajan queued JOB_SUBMISSION cumulative duration</p>
+<img src="karatasks.JOB_SUBMISSION.Queue-duration-histogram.png" />
+
+<pre>
+include(`karatasks.JOB_SUBMISSION.Queue.stats')
+</pre>
+<hr />
+
+<p>Number of tasks in Active state</p>
+<img src="karatasks.JOB_SUBMISSION.Active-total.png" />
+
+<p>karajan active JOB_SUBMISSION tasks</p>
+<img src="karatasks.JOB_SUBMISSION.Active.sorted-start.png" />
+
+<p>karajan active JOB_SUBMISSION cumulative duration</p>
+<img src="karatasks.JOB_SUBMISSION.Active-duration-histogram.png" />
+
+<pre>
+include(`karatasks.JOB_SUBMISSION.Active.stats')
+</pre>
+<hr />
+
+<hr />
+<h2>job clusters</h2>
+1. <img src="karatasks.clusters.png" />  
+<br />
+2. <img src="karatasks.clusters-total.png" /> 
+<br />
+3. <img src="karatasks.clusters.sorted-start.png" />
+<br />
+4. <img src="karatasks.clusters-duration-histogram.png" />
+<br />
+5. <img src="cluster-jobcount-histogram.png" />
+<hr />
+
+<p>end</p>
+</body>

Added: log-processing/libexec/kickstart.html.template
===================================================================
--- log-processing/libexec/kickstart.html.template	                        (rev 0)
+++ log-processing/libexec/kickstart.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -0,0 +1,32 @@
+<html>
+<head><title>Swift log analysis</title></head>
+
+<body><h1>Swift log analysis</h1>
+
+include(`toolbar.html.template')
+
+<h2>kickstart</h2>
+<p>kickstart records.
+(<a href="kickstart.event">text file of events</a>)
+this shows applications which are actually running on a worker node, executing:</p>
+<img src="kickstart.png" />
+
+<p>kickstart records total - this represents how many worker nodes were
+actually executing applications at any time:</p>
+<img src="kickstart-total.png" />
+
+<p>kickstart records sorted by start time:</p>
+<img src="kickstart.sorted-start.png" />
+
+<p>kickstart stats:
+<pre>
+include(`kickstart.stats')
+</pre>
+</p>
+
+<p>cumulative histogram(?) of kickstart job durations</p>
+
+<img src="kickstart-duration-histogram.png" />
+
+<p>end</p>
+</body>

Modified: log-processing/libexec/log-to-karatasks-transitions
===================================================================
--- log-processing/libexec/log-to-karatasks-transitions	2008-10-05 01:51:54 UTC (rev 2269)
+++ log-processing/libexec/log-to-karatasks-transitions	2008-10-05 02:02:52 UTC (rev 2270)
@@ -1,7 +1,7 @@
 #!/bin/bash
 
 # this will take a log file and produce a set of state transition
-# events for every execute2 entity
+# events for every karajan task entity 
 
 # grep filters down to mostly relevant lines first - this is quick
 # reduces the more expensive later processing

Modified: log-processing/libexec/makefile
===================================================================
--- log-processing/libexec/makefile	2008-10-05 01:51:54 UTC (rev 2269)
+++ log-processing/libexec/makefile	2008-10-05 02:02:52 UTC (rev 2270)
@@ -16,7 +16,7 @@
 	cp *.event *.html *.png report-$(SDL)
 
 clean:
-	rm -f start-times.data kickstart-times.data start-time.tmp end-time.tmp threads.list tasks.list log *.data *.shifted *.png *.event *.coloured-event *.total *.tmp *.transitions *.last karatasks-type-counts.txt index.html *.lastsummary execstages.plot total.plot colour.plot jobs-sites.table jobs.retrycount.summary kickstart.stats execution-counts.txt site-duration.txt jobs.retrycount sp.plot karatasks.coloured-sorted-event *.cedps *.stats t.inf *.seenstates tmp-* info.* clusterstats  trname-summary sites-list.data.nm info-md5sums pse2d-tmp.eip
+	rm -f start-times.data kickstart-times.data start-time.tmp end-time.tmp threads.list tasks.list log *.data *.shifted *.png *.event *.coloured-event *.total *.tmp *.transitions *.last karatasks-type-counts.txt index.html *.lastsummary execstages.plot total.plot colour.plot jobs-sites.table jobs.retrycount.summary kickstart.stats execution-counts.txt site-duration.txt jobs.retrycount sp.plot karatasks.coloured-sorted-event *.cedps *.stats t.inf *.seenstates tmp-* clusterstats  trname-summary sites-list.data.nm info-md5sums pse2d-tmp.eip
 
 t.inf: $(LOG)
 	./compute-t-inf > t.inf < $(LOG)
@@ -60,9 +60,6 @@
 execute-start-times.data: log
 	./extract-execute-start-times > execute-start-times.data
 
-all.event: events
-	cat workflow.event execute.event execute2.event karatasks.event kickstart.event > all.event
-
 sites-list.data: execute2.event
 	cat execute2.event | cut -f 6 -d ' ' | sort | uniq > sites-list.data
 

Modified: log-processing/libexec/makefile.webpage
===================================================================
--- log-processing/libexec/makefile.webpage	2008-10-05 01:51:54 UTC (rev 2269)
+++ log-processing/libexec/makefile.webpage	2008-10-05 02:02:52 UTC (rev 2270)
@@ -1,5 +1,8 @@
-webpage: pngs index.html
+webpage: pngs htmls
 
+htmls: index.html execute2.html execute.html karajan.html info.html \
+assorted.html kickstart.html falkon.html scheduler.html
+
 webpage.kara: karatasks.png karatasks-coloured.png karatasks-type-counts.txt \
 karatasks.lastsummary karatasks-coloured-sorted.png \
 karatasks.FILE_TRANSFER.sorted-start.png \
@@ -26,8 +29,6 @@
 
 webpage.weights: weights.png tscores.png
 
-webpage.all: all.png
-
 webpage.info: info.png info.sorted-start.png info-trails.png \
 info-duration-histogram.png info.zeroed-trailsx.png info.stats info-total.png
 
@@ -42,9 +43,12 @@
 dostageout.sorted-start.png dostagein-duration-histogram.png \
 dostageout-duration-histogram.png execute-trails.png execute2-trails.png
 
-index.html: index.html.template execution-counts.txt jobs-sites.table site-duration.txt execute2.lastsummary execute.lastsummary jobs.retrycount.summary trname-summary
-	m4 index.html.template > index.html
+htmldeps: index.html.template execution-counts.txt jobs-sites.table site-duration.txt execute2.lastsummary execute.lastsummary jobs.retrycount.summary trname-summary
 
+%.html: htmldeps
+	m4 $@.template > $@
+
+
 index.html.kara: karatasks.lastsummary karatasks.FILE_OPERATION.stats karatasks.FILE_TRANSFER.stats karatasks.JOB_SUBMISSION.stats karatasks-type-counts.txt 
 
 index.html.kickstart: kickstart.stats

Added: log-processing/libexec/scheduler.html.template
===================================================================
--- log-processing/libexec/scheduler.html.template	                        (rev 0)
+++ log-processing/libexec/scheduler.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -0,0 +1,17 @@
+<html>
+<head><title>Swift log analysis</title></head>
+
+<body><h1>Swift log analysis</h1>
+
+include(`toolbar.html.template')
+
+<hr />
+<h2>scheduler scores</h2>
+<p>Scheduler host scores:</p>
+<img src="weights.png" />
+
+<p>Scheduler load information (tscores):</p>
+<img src="tscores.png" />
+
+<p>end</p>
+</body>

Added: log-processing/libexec/toolbar.html.template
===================================================================
--- log-processing/libexec/toolbar.html.template	                        (rev 0)
+++ log-processing/libexec/toolbar.html.template	2008-10-05 02:02:52 UTC (rev 2270)
@@ -0,0 +1,13 @@
+<p>
+<a href="index.html">Summary</a> |
+Channels:
+<a href="execute.html">execute</a> |
+<a href="execute2.html">execute2</a> |
+<a href="karajan.html">karajan</a> |
+<a href="kickstart.html">kickstart</a> |
+<a href="info.html">info</a> |
+<a href="falkon.html">falkon</a> |
+<a href="assorted.html">assorted swift library routines</a>
+| Other:
+<a href="scheduler.html">scheduler</a>
+</p>



From noreply at svn.ci.uchicago.edu  Sun Oct  5 14:30:56 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sun,  5 Oct 2008 14:30:56 -0500 (CDT)
Subject: [Swift-commit] r2271 - SwiftApps
Message-ID: <20081005193056.66335228035@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-05 14:30:54 -0500 (Sun, 05 Oct 2008)
New Revision: 2271

Added:
   SwiftApps/adem/
Log:
Making Adem in SwiftApps



From noreply at svn.ci.uchicago.edu  Sun Oct  5 14:33:58 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sun,  5 Oct 2008 14:33:58 -0500 (CDT)
Subject: [Swift-commit] r2273 - /
Message-ID: <20081005193358.60283228159@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-05 14:33:58 -0500 (Sun, 05 Oct 2008)
New Revision: 2273

Removed:
   adem/
Log:
deleting adem from root



From noreply at svn.ci.uchicago.edu  Sun Oct  5 16:40:36 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sun,  5 Oct 2008 16:40:36 -0500 (CDT)
Subject: [Swift-commit] r2274 - SwiftApps/adem
Message-ID: <20081005214036.65A40228185@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-05 16:40:35 -0500 (Sun, 05 Oct 2008)
New Revision: 2274

Added:
   SwiftApps/adem/nab-5.1.2.tar.gz
Log:
New import test

Added: SwiftApps/adem/nab-5.1.2.tar.gz
===================================================================
(Binary files differ)


Property changes on: SwiftApps/adem/nab-5.1.2.tar.gz
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream



From noreply at svn.ci.uchicago.edu  Sun Oct  5 21:37:26 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sun,  5 Oct 2008 21:37:26 -0500 (CDT)
Subject: [Swift-commit] r2277 - log-processing/libexec
Message-ID: <20081006023726.ECDA8228035@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-05 21:37:26 -0500 (Sun, 05 Oct 2008)
New Revision: 2277

Modified:
   log-processing/libexec/execstages-plot
   log-processing/libexec/execstages.plot.template
   log-processing/libexec/makefile
Log:
remove execute events from summary graph - they often do not fit on the same scale as execute2/stagein/stageout events

Modified: log-processing/libexec/execstages-plot
===================================================================
--- log-processing/libexec/execstages-plot	2008-10-05 23:17:12 UTC (rev 2276)
+++ log-processing/libexec/execstages-plot	2008-10-06 02:37:26 UTC (rev 2277)
@@ -6,9 +6,6 @@
 ./events-in-progress < dostagein.event > esp.dostagein.a.tmp
 ./normalise-event-start-time < esp.dostagein.a.tmp > esp.dostagein.tmp
 
-./events-in-progress < execute.event > esp.execute.a.tmp
-./normalise-event-start-time < esp.execute.a.tmp > esp.execute.tmp
-
 ./events-in-progress < dostageout.event > esp.dostageout.a.tmp
 ./normalise-event-start-time < esp.dostageout.a.tmp > esp.dostageout.tmp
 

Modified: log-processing/libexec/execstages.plot.template
===================================================================
--- log-processing/libexec/execstages.plot.template	2008-10-05 23:17:12 UTC (rev 2276)
+++ log-processing/libexec/execstages.plot.template	2008-10-06 02:37:26 UTC (rev 2277)
@@ -6,13 +6,12 @@
 set style arrow 1 nohead lt 1
 set style arrow 2 nohead lt 2 
 set style arrow 3 nohead lt 3 
-set style arrow 4 nohead lt 4 
 set border 0
 
 set xrange [0:MAXTIME]
 
 plot 'esp.execute2.tmp' with vector arrowstyle 1 title 'execute2', \
      'esp.dostagein.tmp' with vector arrowstyle 2 title 'dostagein', \
-     'esp.execute.tmp' with vector arrowstyle 3 title 'execute', \
-     'esp.dostageout.tmp' with vector arrowstyle 4 title 'dostageout'
+     'esp.dostageout.tmp' with vector arrowstyle 3 title 'dostageout'
 
+

Modified: log-processing/libexec/makefile
===================================================================
--- log-processing/libexec/makefile	2008-10-05 23:17:12 UTC (rev 2276)
+++ log-processing/libexec/makefile	2008-10-06 02:37:26 UTC (rev 2277)
@@ -101,7 +101,7 @@
 jobs.retrycount.summary: execute2.event
 	cat execute2.event  | cut -f 5 -d ' ' | sort | uniq -c | sort | sed 's/^ *\(.*\) .*$$/\1/' | uniq -c > jobs.retrycount.summary
 
-execstages.png: execute2.event dostagein.event execute.event dostageout.event start-time.tmp
+execstages.png: execute2.event dostagein.event dostageout.event start-time.tmp
 	./execstages-plot
 
 # weight processing



From noreply at svn.ci.uchicago.edu  Sun Oct  5 21:47:05 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sun,  5 Oct 2008 21:47:05 -0500 (CDT)
Subject: [Swift-commit] r2278 - log-processing/libexec
Message-ID: <20081006024705.9EF95228185@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-05 21:47:04 -0500 (Sun, 05 Oct 2008)
New Revision: 2278

Added:
   log-processing/libexec/all-logs-active-jobsubmissions-count-graph
   log-processing/libexec/everylog-active-submissions.plot
Log:
script to plot every karajan ACTIVE JOB_SUBMISSION from a directory tree of log files

Added: log-processing/libexec/all-logs-active-jobsubmissions-count-graph
===================================================================
--- log-processing/libexec/all-logs-active-jobsubmissions-count-graph	                        (rev 0)
+++ log-processing/libexec/all-logs-active-jobsubmissions-count-graph	2008-10-06 02:47:04 UTC (rev 2278)
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+find /Users/benc/work/logs/ -type f -name \*.log -exec grep 'DEBUG TaskImpl Task(type=JOB_SUBMISSION, identity=' {} \; | grep 'setting status to Active' | ./iso-to-secs | cut -d ' ' -f 1 | sort -n | ./number-sites-list  > all-logs-active-tasks.data.1.tmp
+
+while read l r ; do
+  echo $(date -r $l +"%Y-%m-%d") $r
+done < all-logs-active-tasks.data.1.tmp > all-logs-active-tasks.data
+
+gnuplot everylog-active-submissions.plot
+


Property changes on: log-processing/libexec/all-logs-active-jobsubmissions-count-graph
___________________________________________________________________
Name: svn:executable
   + *

Added: log-processing/libexec/everylog-active-submissions.plot
===================================================================
--- log-processing/libexec/everylog-active-submissions.plot	                        (rev 0)
+++ log-processing/libexec/everylog-active-submissions.plot	2008-10-06 02:47:04 UTC (rev 2278)
@@ -0,0 +1,13 @@
+set terminal png
+set output 'all-logs-active-tasks.png'
+
+set ylabel 'cumulative invocations'
+set xlabel 'time (s)'
+
+set timefmt '%Y-%m-%d'
+set xdata time
+
+set key outside below
+
+plot 'all-logs-active-tasks.data' using 1:2 with lines
+



From noreply at svn.ci.uchicago.edu  Sun Oct  5 22:47:05 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sun,  5 Oct 2008 22:47:05 -0500 (CDT)
Subject: [Swift-commit] r2279 - log-processing/libexec
Message-ID: <20081006034705.4E1C3228192@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-05 22:47:04 -0500 (Sun, 05 Oct 2008)
New Revision: 2279

Added:
   log-processing/libexec/sec-to-utc-day
Modified:
   log-processing/libexec/all-logs-active-jobsubmissions-count-graph
Log:
more portable handling of dates - date -r has different meanings in bsd-land and gnu-land

Modified: log-processing/libexec/all-logs-active-jobsubmissions-count-graph
===================================================================
--- log-processing/libexec/all-logs-active-jobsubmissions-count-graph	2008-10-06 02:47:04 UTC (rev 2278)
+++ log-processing/libexec/all-logs-active-jobsubmissions-count-graph	2008-10-06 03:47:04 UTC (rev 2279)
@@ -3,7 +3,7 @@
 find /Users/benc/work/logs/ -type f -name \*.log -exec grep 'DEBUG TaskImpl Task(type=JOB_SUBMISSION, identity=' {} \; | grep 'setting status to Active' | ./iso-to-secs | cut -d ' ' -f 1 | sort -n | ./number-sites-list  > all-logs-active-tasks.data.1.tmp
 
 while read l r ; do
-  echo $(date -r $l +"%Y-%m-%d") $r
+  echo $(./sec-to-utc-day $l) $r
 done < all-logs-active-tasks.data.1.tmp > all-logs-active-tasks.data
 
 gnuplot everylog-active-submissions.plot

Copied: log-processing/libexec/sec-to-utc-day (from rev 2267, log-processing/libexec/sec-to-utc)
===================================================================
--- log-processing/libexec/sec-to-utc-day	                        (rev 0)
+++ log-processing/libexec/sec-to-utc-day	2008-10-06 03:47:04 UTC (rev 2279)
@@ -0,0 +1,8 @@
+#!/usr/bin/perl
+
+($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = gmtime($ARGV[0]);
+
+# print 2007-06-12
+
+printf("%d-%02d-%02d", $year+1900, $mon+1, $mday);
+



From noreply at svn.ci.uchicago.edu  Mon Oct  6 00:27:55 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  6 Oct 2008 00:27:55 -0500 (CDT)
Subject: [Swift-commit] r2280 - log-processing/libexec
Message-ID: <20081006052755.96E3B228192@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-06 00:27:54 -0500 (Mon, 06 Oct 2008)
New Revision: 2280

Modified:
   log-processing/libexec/all-logs-active-jobsubmissions-count-graph
Log:
take log repo path on command line

Modified: log-processing/libexec/all-logs-active-jobsubmissions-count-graph
===================================================================
--- log-processing/libexec/all-logs-active-jobsubmissions-count-graph	2008-10-06 03:47:04 UTC (rev 2279)
+++ log-processing/libexec/all-logs-active-jobsubmissions-count-graph	2008-10-06 05:27:54 UTC (rev 2280)
@@ -1,6 +1,6 @@
 #!/bin/bash
 
-find /Users/benc/work/logs/ -type f -name \*.log -exec grep 'DEBUG TaskImpl Task(type=JOB_SUBMISSION, identity=' {} \; | grep 'setting status to Active' | ./iso-to-secs | cut -d ' ' -f 1 | sort -n | ./number-sites-list  > all-logs-active-tasks.data.1.tmp
+find $1 -type f -name \*.log -exec grep 'DEBUG TaskImpl Task(type=JOB_SUBMISSION, identity=' {} \; | grep 'setting status to Active' | ./iso-to-secs | cut -d ' ' -f 1 | sort -n | ./number-sites-list  > all-logs-active-tasks.data.1.tmp
 
 while read l r ; do
   echo $(./sec-to-utc-day $l) $r



From noreply at svn.ci.uchicago.edu  Mon Oct  6 11:44:36 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  6 Oct 2008 11:44:36 -0500 (CDT)
Subject: [Swift-commit] r2281 - log-processing/libexec
Message-ID: <20081006164436.2C50F228185@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-06 11:44:34 -0500 (Mon, 06 Oct 2008)
New Revision: 2281

Added:
   log-processing/libexec/info-and-karajan-actives
   log-processing/libexec/info-and-karajan-actives.2.plot
   log-processing/libexec/info-and-karajan-actives.plot
Modified:
   log-processing/libexec/info.html.template
   log-processing/libexec/makefile.webpage
Log:
plots of active status vs info log start/end times

Added: log-processing/libexec/info-and-karajan-actives
===================================================================
--- log-processing/libexec/info-and-karajan-actives	                        (rev 0)
+++ log-processing/libexec/info-and-karajan-actives	2008-10-06 16:44:34 UTC (rev 2281)
@@ -0,0 +1,51 @@
+#!/bin/bash
+
+# for every job which has an info file,
+# create two columns, one being the start time of the job according
+# to the info file and the other being the Active time according to
+# karajan
+
+
+# for each info file
+
+for infofn in $1/*-info; do
+
+#   get info id
+
+  jobid=$(echo $infofn | sed 's%^.*/\([^/]*\)-info$%\1%' )
+#  echo Processing info file for job $jobid in file $infofn >&2
+# Progress  2007-10-29 10:05:50+0000  RM_JOBDIR
+INFOSTART=$(grep -E '^Progress ' $infofn | sed "s/^Progress *\([^ ]* [^ ]*\) *\([^ ]*\).*/\1 $jobid \2/" | head -n 1 | ./iso-to-secs | cut -d ' ' -f 1)
+INFOEND=$(grep -E '^Progress ' $infofn | sed "s/^Progress *\([^ ]* [^ ]*\) *\([^ ]*\).*/\1 $jobid \2/" | tail -n 1 | ./iso-to-secs | cut -d ' ' -f 1)
+
+#   get karajan ID
+# execute2.event lines look like this:
+# 1223271839.650 0.782999992370605 touch-a7kvof0j JOB_END 0-0-4-1 localhost 97kvof0j
+
+threadid=$(grep -E "^.* .* $jobid .* .* .* .*$" execute2.event | sed 's/^.* .* .* .* \(.*\) .* .*$/\1/')
+
+#   get karajan Active timestamp for this
+#1223271840.424 0-0-4-1-1223271839341 Submitting FILE_OPERATION 
+
+KARASTART=$(grep JOB_SUBMISSION karatasks.transitions | grep Active | grep -E "^.* ${threadid}-.* .* .*" | cut -d ' ' -f 1)
+KARAEND=$(grep JOB_SUBMISSION karatasks.transitions | grep Completed | grep -E "^.* ${threadid}-.* .* .*" | cut -d ' ' -f 1)
+
+#   output
+
+DIFFSTART=$(echo $INFOSTART - $KARASTART | bc -l )
+DIFFEND=$(echo $INFOEND - $KARAEND | bc -l )
+
+echo $INFOSTART $KARASTART $DIFFSTART >> info-and-karajan-actives.data
+echo $INFOEND $KARAEND $DIFFEND >> info-and-karajan-actives.end.data
+
+
+done
+
+gnuplot info-and-karajan-actives.plot
+
+cat info-and-karajan-actives.data | cut -d ' ' -f 3 | sort -n | ./number-sites-list > info-and-karajan-actives.2.data
+cat info-and-karajan-actives.end.data | cut -d ' ' -f 3 | sort -n | ./number-sites-list > info-and-karajan-actives.end.2.data
+
+gnuplot info-and-karajan-actives.2.plot
+
+


Property changes on: log-processing/libexec/info-and-karajan-actives
___________________________________________________________________
Name: svn:executable
   + *

Added: log-processing/libexec/info-and-karajan-actives.2.plot
===================================================================
--- log-processing/libexec/info-and-karajan-actives.2.plot	                        (rev 0)
+++ log-processing/libexec/info-and-karajan-actives.2.plot	2008-10-06 16:44:34 UTC (rev 2281)
@@ -0,0 +1,10 @@
+set terminal png
+set output 'info-and-karajan-actives.2.png'
+
+set xlabel 'offset from Active notification'
+set ylabel 'cumulative events below this offset'
+
+set key outside below
+
+plot 'info-and-karajan-actives.2.data' using 1:2 with lines title 'start time (info - karajan)', 'info-and-karajan-actives.end.2.data' using 1:2 with lines title 'end time (info - karajan)'
+

Added: log-processing/libexec/info-and-karajan-actives.plot
===================================================================
--- log-processing/libexec/info-and-karajan-actives.plot	                        (rev 0)
+++ log-processing/libexec/info-and-karajan-actives.plot	2008-10-06 16:44:34 UTC (rev 2281)
@@ -0,0 +1,10 @@
+set terminal png
+set output 'info-and-karajan-actives.png'
+
+set ylabel 'offset from Active notification'
+set xlabel 'time of start according to info'
+
+set key outside below
+
+plot 'info-and-karajan-actives.data' using 1:3 title 'start times (info - karajan)', 'info-and-karajan-actives.end.data' using 1:3 title 'end times (info - karajan)'
+

Modified: log-processing/libexec/info.html.template
===================================================================
--- log-processing/libexec/info.html.template	2008-10-06 05:27:54 UTC (rev 2280)
+++ log-processing/libexec/info.html.template	2008-10-06 16:44:34 UTC (rev 2281)
@@ -24,6 +24,13 @@
 <img src="info-total.png" />
 
 <hr />
+
+<p>Offsets between job submission Active events and start times reported
+by info.</p>
+<img src="info-and-karajan-actives.png" />
+<br/>
+<img src="info-and-karajan-actives.2.png" />
+
 <p>end</p>
 </body>
 </html>

Modified: log-processing/libexec/makefile.webpage
===================================================================
--- log-processing/libexec/makefile.webpage	2008-10-06 05:27:54 UTC (rev 2280)
+++ log-processing/libexec/makefile.webpage	2008-10-06 16:44:34 UTC (rev 2281)
@@ -41,7 +41,8 @@
 initshareddir.png createdirset.png createdirset-total.png dostagein.png \
 dostagein-total.png dostageout.png dostageout-total.png execstages.png \
 dostageout.sorted-start.png dostagein-duration-histogram.png \
-dostageout-duration-histogram.png execute-trails.png execute2-trails.png
+dostageout-duration-histogram.png execute-trails.png execute2-trails.png \
+info-and-karajan-actives.png info-and-karajan-actives.2.png
 
 htmldeps: index.html.template execution-counts.txt jobs-sites.table site-duration.txt execute2.lastsummary execute.lastsummary jobs.retrycount.summary trname-summary
 
@@ -53,3 +54,5 @@
 
 index.html.kickstart: kickstart.stats
 
+info-and-karajan-actives.png info-and-karajan-actives.2.png: execute2.event karatasks.transitions
+	./info-and-karajan-actives $(IDIR)



From noreply at svn.ci.uchicago.edu  Mon Oct  6 20:42:23 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  6 Oct 2008 20:42:23 -0500 (CDT)
Subject: [Swift-commit] r2282 - log-processing/libexec
Message-ID: <20081007014223.571232281B4@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-06 20:42:22 -0500 (Mon, 06 Oct 2008)
New Revision: 2282

Modified:
   log-processing/libexec/everylog-vs-versions
Log:
fix a hardcoded path that should be taken from command line

Modified: log-processing/libexec/everylog-vs-versions
===================================================================
--- log-processing/libexec/everylog-vs-versions	2008-10-06 16:44:34 UTC (rev 2281)
+++ log-processing/libexec/everylog-vs-versions	2008-10-07 01:42:22 UTC (rev 2282)
@@ -14,6 +14,6 @@
 
 find $1 -name \*.log -and -not -name cluster-\*  -exec basename {} \; | sed 's/^\(.*\)-........-....-........\.log$/\1/' | sort | uniq -c | sort -nr > everylog-basenames-scores.txt
 
-./everylog-durations-of-workflows ~/work/everylog/ | sort -k 2 > tmp-evl-duration
+./everylog-durations-of-workflows $1 | sort -k 2 > tmp-evl-duration
 ./sum-tt1.pl < tmp-evl-duration  | sort -r -n -k 2 > everylog-cumulative-durations-per-basename.txt
 



From noreply at svn.ci.uchicago.edu  Mon Oct  6 20:42:50 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  6 Oct 2008 20:42:50 -0500 (CDT)
Subject: [Swift-commit] r2283 - log-processing/libexec
Message-ID: <20081007014250.D2EF12281B4@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-06 20:42:50 -0500 (Mon, 06 Oct 2008)
New Revision: 2283

Modified:
   log-processing/libexec/everylog-active-submissions.plot
Log:
more human readable date format on x axis

Modified: log-processing/libexec/everylog-active-submissions.plot
===================================================================
--- log-processing/libexec/everylog-active-submissions.plot	2008-10-07 01:42:22 UTC (rev 2282)
+++ log-processing/libexec/everylog-active-submissions.plot	2008-10-07 01:42:50 UTC (rev 2283)
@@ -9,5 +9,7 @@
 
 set key outside below
 
+set format x "%b %y"
+
 plot 'all-logs-active-tasks.data' using 1:2 with lines
 



From noreply at svn.ci.uchicago.edu  Mon Oct  6 21:33:23 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  6 Oct 2008 21:33:23 -0500 (CDT)
Subject: [Swift-commit] r2284 - log-processing/libexec
Message-ID: <20081007023323.831FA2281B7@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-06 21:33:23 -0500 (Mon, 06 Oct 2008)
New Revision: 2284

Modified:
   log-processing/libexec/everylog-time-cumulative.plot
   log-processing/libexec/everylog-vs-versions
Log:
better human readable date stamps

Modified: log-processing/libexec/everylog-time-cumulative.plot
===================================================================
--- log-processing/libexec/everylog-time-cumulative.plot	2008-10-07 01:42:50 UTC (rev 2283)
+++ log-processing/libexec/everylog-time-cumulative.plot	2008-10-07 02:33:23 UTC (rev 2284)
@@ -4,7 +4,13 @@
 set ylabel 'cumulative invocations'
 set xlabel 'time'
 
+set timefmt '%Y-%m-%d'
+set xdata time
+
+set format x "%b %y"
+
+
 set key outside below
 
-plot 'everylog-time-cumulative.data' with lines
+plot 'everylog-time-cumulative.data' using 1:2 with lines
 

Modified: log-processing/libexec/everylog-vs-versions
===================================================================
--- log-processing/libexec/everylog-vs-versions	2008-10-07 01:42:50 UTC (rev 2283)
+++ log-processing/libexec/everylog-vs-versions	2008-10-07 02:33:23 UTC (rev 2284)
@@ -8,7 +8,7 @@
 
 gnuplot everylog-version-cumulative.plot
 
-cat everylog-vs-versions.data | cut -f 1 -d ' ' | sort -n | ./number-sites-list  > everylog-time-cumulative.data
+cat everylog-vs-versions.data | cut -f 1 -d ' ' | sort -n | ./number-sites-list  | while read l r ; do  echo $(./sec-to-utc-day $l) $r ; done > everylog-time-cumulative.data
 
 gnuplot everylog-time-cumulative.plot
 



From noreply at svn.ci.uchicago.edu  Tue Oct  7 09:55:44 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue,  7 Oct 2008 09:55:44 -0500 (CDT)
Subject: [Swift-commit] r2285 - log-processing/libexec
Message-ID: <20081007145544.B30F02281BB@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-07 09:55:43 -0500 (Tue, 07 Oct 2008)
New Revision: 2285

Modified:
   log-processing/libexec/info-and-karajan-actives
Log:
normalise seconds on info-vs-karajan plot to start at 0

Modified: log-processing/libexec/info-and-karajan-actives
===================================================================
--- log-processing/libexec/info-and-karajan-actives	2008-10-07 02:33:23 UTC (rev 2284)
+++ log-processing/libexec/info-and-karajan-actives	2008-10-07 14:55:43 UTC (rev 2285)
@@ -35,12 +35,15 @@
 DIFFSTART=$(echo $INFOSTART - $KARASTART | bc -l )
 DIFFEND=$(echo $INFOEND - $KARAEND | bc -l )
 
-echo $INFOSTART $KARASTART $DIFFSTART >> info-and-karajan-actives.data
-echo $INFOEND $KARAEND $DIFFEND >> info-and-karajan-actives.end.data
+echo $INFOSTART $KARASTART $DIFFSTART >> info-and-karajan-actives.3.data
+echo $INFOEND $KARAEND $DIFFEND >> info-and-karajan-actives.end.3.data
 
 
 done
 
+./normalise-event-start-time < info-and-karajan-actives.3.data > info-and-karajan-actives.data
+./normalise-event-start-time < info-and-karajan-actives.end.3.data > info-and-karajan-actives.end.data
+
 gnuplot info-and-karajan-actives.plot
 
 cat info-and-karajan-actives.data | cut -d ' ' -f 3 | sort -n | ./number-sites-list > info-and-karajan-actives.2.data



From noreply at svn.ci.uchicago.edu  Fri Oct 10 05:15:05 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 10 Oct 2008 05:15:05 -0500 (CDT)
Subject: [Swift-commit] r2286 - trunk/libexec
Message-ID: <20081010101505.F171822807D@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-10 05:15:05 -0500 (Fri, 10 Oct 2008)
New Revision: 2286

Modified:
   trunk/libexec/wrapper.sh
Log:
off-by-one error, introduced in r2226. this was breaking things in the case of tguc + coasters + gram4 (but not gram2).

Modified: trunk/libexec/wrapper.sh
===================================================================
--- trunk/libexec/wrapper.sh	2008-10-07 14:55:43 UTC (rev 2285)
+++ trunk/libexec/wrapper.sh	2008-10-10 10:15:05 UTC (rev 2286)
@@ -163,7 +163,7 @@
 export PATH=$PATHPREFIX:$PATH
 fi
 
-if [ "X${EXEC:1:1}" != "X/" ] ; then
+if [ "X${EXEC:0:1}" != "X/" ] ; then
 export EXEC=$(which $EXEC)
 fi
 



From noreply at svn.ci.uchicago.edu  Sat Oct 11 04:06:27 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sat, 11 Oct 2008 04:06:27 -0500 (CDT)
Subject: [Swift-commit] r2287 - trunk/docs
Message-ID: <20081011090627.BA4C42281BB@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-11 04:06:25 -0500 (Sat, 11 Oct 2008)
New Revision: 2287

Modified:
   trunk/docs/userguide.xml
Log:
notes on coasters; more removal of Workflow references. a bit of other fiddling

Modified: trunk/docs/userguide.xml
===================================================================
--- trunk/docs/userguide.xml	2008-10-10 10:15:05 UTC (rev 2286)
+++ trunk/docs/userguide.xml	2008-10-11 09:06:25 UTC (rev 2287)
@@ -19,7 +19,7 @@
 	<para>
 Swift is a data-oriented coarse grained scripting language that
 supports dataset typing and mapping, dataset iteration,
-conditional branching, and sub-workflow composition.
+conditional branching, and procedural composition.
 	</para>
 	<para>
 Swift programs (or <firstterm>workflows</firstterm>) are written in
@@ -1196,15 +1196,15 @@
 								report the first error encountered and
 								immediately stop execution. If set to
 								<constant>true</constant>, Swift will attempt to
-								run as much as possible from a workflow before
+								run as much as possible from a SwiftScript program before
 								stopping execution and reporting all errors
 								encountered.
 							</para>
-							<para>When developing workflows, using the
+							<para>When developing SwiftScript programs, using the
 								default value of <constant>false</constant> can
-								make the workflow easier to debug. However
+								make the program easier to debug. However
 								in production runs, using <constant>true</constant>
-								will allow more of a workflow to be run before
+								will allow more of a SwiftScript program to be run before
 								Swift aborts execution.
 							</para>
 						</listitem>
@@ -1296,10 +1296,10 @@
 							
 								Swift can generate a 
 <ulink url="http://www.graphviz.org/">Graphviz</ulink> file representing 
-								the structure of the workflow it runs. If this
+								the structure of the SwiftScript program it has run. If this
 								property is set to <literal>true</literal>,
 								Swift will save the provenance graph in a file
-								named by concatenating the workflow name and the
+								named by concatenating the program name and the
 								instance ID (e.g. <filename
 								class="file">helloworld-ht0adgi315l61.dot</filename>). 
 							</para>
@@ -1551,7 +1551,7 @@
 							<para>
 							
 								Limits the number of concurrent submissions for
-								a workflow instance. This throttle only limits
+								a run. This throttle only limits
 								the number of concurrent tasks (jobs) that are
 								being sent to sites, not the total number of
 								concurrent jobs that can be run. The submission
@@ -1714,7 +1714,7 @@
 
 							<para>
 Indicates whether the working directory on the remote site should be
-left intact even when the workflow completes successfully. This can be
+left intact even when a run completes successfully. This can be
 used to inspect the site working directory for debugging purposes.
 							</para>
 						</listitem>
@@ -1755,12 +1755,12 @@
   </varlistentry>
   <varlistentry><term>-typecheck</term>
     <listitem><para>
-      Does a typecheck instead of executing the workflow </para></listitem>
+      Does a typecheck of a SwiftScript program, instead of executing it.</para></listitem>
   </varlistentry>
 
   <varlistentry><term>-dryrun</term>
     <listitem><para>
-      Runs the workflow without submitting any jobs (can be used to get
+      Runs the SwiftScript program without submitting any jobs (can be used to get
       a graph)
     </para></listitem>
   </varlistentry>
@@ -1803,16 +1803,16 @@
   <varlistentry><term>-logfile <literal>file</literal></term>
     <listitem><para>
       Specifies a file where log messages should go to. By default 
-      Swift uses the name of the workflow being run and a numeric index
+      Swift uses the name of the program being run and a numeric index
       (e.g. myworkflow.1.log) 
     </para></listitem>
   </varlistentry>
 
   <varlistentry><term>-runid <literal>identifier</literal></term>
     <listitem><para>
-Specifies the run identifier. This must be unique for every invocation of a workflow and is used in several places to keep files from different executions cleanly separated. By default, a datestamp and random number are used to generate a run identifier. When using this parameter, care should be taken to ensure
+Specifies the run identifier. This must be unique for every invocation and is used in several places to keep files from different executions cleanly separated. By default, a datestamp and random number are used to generate a run identifier. When using this parameter, care should be taken to ensure
 that the run ID remains unique with respect to all other run IDs that might
-be used, irrespective of (at least) expected run location, workflow or user.
+be used, irrespective of (at least) expected execution sites, program or user.
     </para></listitem>
   </varlistentry>
 
@@ -1857,7 +1857,7 @@
 <itemizedlist>
 <listitem>0 - success</listitem>
 <listitem>1 - command line syntax error or missing project name</listitem>
-<listitem>2 - error during workflow execution</listitem>
+<listitem>2 - error during execution</listitem>
 <listitem>3 - error during compilation of SwiftScript program</listitem>
 <listitem>4 - input file does not exist</listitem>
 </itemizedlist>
@@ -1921,30 +1921,30 @@
 
 	<section id="restart"> <title>Workflow restart/recovery</title>
 		<para>
-If a workflow fails, Swift can resume that workflow from the point of
-failure. If a Swift workflow fails, a restart log file will be generated
-using the unique job ID, with a .rlog extension. This restart log
+If a run fails, Swift can resume the program from the point of
+failure. When a run fails, a restart log file will be left behind in
+a file named using the unique job ID and a .rlog extension. This restart log
 can then be passed to a subsequent Swift invocation using the -resume
-parameter. Swift will resume executing the workflow. Previously executed
-tasks will not be run a second time. The SwiftScript source file should
-not be modified between invocations.
+parameter. Swift will resume execution, avoiding execution of invocations
+that have previously completed successfully. The SwiftScript source file
+and input data files should not be modified between runs.
 		</para>
 		<para>
-Every execution of a workflow creates a restart
+Every run creates a restart
 log file with a named composed of the file name of the workflow
 being executed, an invocation ID, a numeric ID, and the <filename
 class="file">.rlog</filename> extension. For example, <filename
 class="file">example.swift</filename>, when executed, could produce
 the following restart log file: <filename
 class="file">example-ht0adgi315l61.0.rlog</filename>. Normally, if
-the workflow completes successfully, the restart log file is
+the run completes successfully, the restart log file is
 deleted. If however the workflow fails, <command>swift</command>
-can use the restart log file to continue the
-execution of the workflow from a point before the
-failure occurred. In order to restart a workflow from a restart log
+can use the restart log file to continue
+execution from a point before the
+failure occurred. In order to restart from a restart log
 file, the <option>-resume <parameter><filename
 class="file">logfile</filename></parameter></option> argument can be
-used after the compiled workflow file name. Example:
+used after the SwiftScript program file name. Example:
 
 <screen>
 <prompt>&gt;</prompt> <command>swift</command> <option>-resume <filename
@@ -2069,9 +2069,9 @@
 libexec/scheduler.xml and etc/karajan.properties to refer to the new
 scheduler.
 </para>
-<para>Execution providers, which allow to Swift to execute jobs through
-specific mechanisms (such as local fork or through GRAM) can be
-implemented as Java CoG kit providers.
+<para>Execution providers and filesystem providers, which allow to Swift
+to execute jobs and to stage files in and out through mechanisms such
+as GRAM and GridFTP can be implemented as Java CoG kit providers.
 </para>
 	</section>
 
@@ -2296,7 +2296,7 @@
 			</para>
 			<para id="profile.globus.coastersPerNode">coastersPerNode specifies
 the number of coaster workers to be run on each node. This profile entry
-is used by the coaster provider.
+is used by the <link linkend="coasters">coaster execution provider</link>.
 			</para>
 		</section>
 		<section id="profile.env"><title>env namespace</title>
@@ -2311,7 +2311,9 @@
 		<para>
 Swift can group a number of short job submissions into a single larger
 job submission to minimize overhead involved in launching jobs (for example,
-caused by security negotiation and queuing delay).
+caused by security negotiation and queuing delay). In general,
+<link linkend="coasters">CoG coasters</link> should be used in preference
+to the clustering mechanism documented in this section.
 		</para>
 
 		<para>
@@ -2358,7 +2360,9 @@
 records into a directory at the CI for reporting, usage tracking, support and debugging.
 			</para>
 			<para>
+<screen>
 rsync --ignore-existing *.log *.d login.ci.uchicago.edu:/home/benc/swift-logs/ --verbose
+</screen>
 			</para>
 		</section>
 		<section><title>Specifying TeraGrid allocations</title>
@@ -2461,7 +2465,7 @@
 
 		<para>
 Transfer methods are specified with the &lt;gridftp&gt; element or
-with the &lt;filesystem&gt; method.
+with the &lt;filesystem&gt; element.
 		</para>
 		<para>
 To use gridftp or local filesystem copy, use the &lt;gridftp&gt;
@@ -2483,7 +2487,10 @@
 For additional ssh configuration information, see the ssh execution
 provider documentation below.
 		</para>
-
+Filesystem access using <link linkend="coasters">CoG coasters</link> can be
+also be specified using the &lt;filesystem&gt; element. More detail about
+configuring that can be found in the <link linkend="coasters">CoG
+coasters</link> section.
 </section>
 
 <section><title>Execution method specification</title>
@@ -2548,16 +2555,15 @@
 </screen>
 		</para>
 		<para>
-For execution using the CoG Coaster mechanism, the coaster provider
+For execution using the 
+<link linkend="coasters">CoG Coaster mechanism</link>, the coaster provider
 should be used:
 <screen>
 &lt;execution provider="coaster" url="tg-grid.uc.teragrid.org"
     jobmanager="gt2:gt2:pbs" />
 </screen>
-with the jobmanager parameter specifying: the cog provider to use to
-submit the coaster head job; the cog provider to use to submit
-coaster worker jobs; and optionally the jobmanager to be used by
-worker submission.
+More details about configuration of coasters can be found in the
+<link linkend="coasters">section on coasters</link>.
 		</para>
 </section>
 <section><title>Other site catalog parameters</title>
@@ -2584,7 +2590,8 @@
 		</section>
 
 		<para>
-The site catalog is an evolution of the VDS site catalog which is documented
+The site catalog format is an evolution of the VDS site catalog format which
+is documented
 <ulink url="http://vds.uchicago.edu/vds/doc/userguide/html/H_SiteCatalog.html">here</ulink>.
 		</para>
 	</section>
@@ -2601,7 +2608,7 @@
 		</para>
 		<para>
 The format is one line per executable per site, with fields separated by
-tabs. Spaces cannot be used as a field separator.
+tabs. Spaces cannot be used to separate fields.
 		</para>
 		<para>Some example entries:
 <screen>
@@ -2680,7 +2687,8 @@
 with-provider-condor - build with CoG condor provider
 		</para>
 		<para>
-with-provider-coaster - build with CoG coaster provider
+with-provider-coaster - build with CoG coaster provider (see
+<link linkend="coasters">the section on coasters</link>)
 		</para>
 		<para>
 with-provider-deef - build with Falkon provider deef. In order for this
@@ -2723,5 +2731,66 @@
 </screen>
 		</para>
 	</section>
+	<section id="coasters"><title>Coasters</title>
+<para>Coasters were introduced in Swift v0.6 as an experimental feature.
+</para>
+<para>
+In many applications, Swift performance can be greatly enhanced by the
+use of CoG coasters. CoG coasters provide a low-overhead job submission
+and file transfer mechanism suited for the execution of short jobs
+(on the order of a few seconds) and the transfer of small files (on the
+order of a few kilobytes) for which other grid protocols such as GRAM
+and GridFTP are poorly suited.
+</para>
+<para>
+The coaster mechanism submits a head job using some other execution
+mechanism such as GRAM, and for each worker node that will be used in
+a remote cluster, it submits a worker job, again using some other
+execution mechanism such as GRAM.
+</para>
+<para>
+The head job manages file transfers and the dispatch of execution jobs
+to workers. Much of the overhead associated with other grid protocols
+(such as authentication and authorization, and allocation of worker nodes
+by the site's local resource manager) is reduced, because that overhead
+is associated with the allocation of a coaster head or coaster worker,
+rather than with every Swift-level procedure invocation; potentially hundreds
+or thousands of Swift-level procedure invocations can be run through a single
+worker.
+</para>
+<para>
+Coasters can be configured for use in two situations: job execution and
+file transfer.
+</para>
+<para>
+To use for job execution, specify a sites.xml execution element like this:
+<screen>
+&lt;execution provider="coaster" jobmanager="gt2:gt2:pbs" url="grid.myhost.org"&gt;
+</screen>
+The jobmanager string contains more detail than with other providers. It
+contains either two or three colon separated fields:
+1:the provider to be use to execute the coaster head job - this provider
+will submit from the Swift client side environment. Commonly this will be
+one of the GRAM providers; 2: the provider
+to be used to execute coaster worker jobs. This provider will be used
+to submit from the coaster head job environment, so a local scheduler
+provider can sometimes be used instead of GRAM. 3: optionally, the
+jobmanager to be used when submitting worker job using the provider
+specified in field 2.
+</para>
+<para>
+To use for file transfer, specify a sites.xml filesystem element like this:
+<screen>
+&lt;filesystem url="gt2://grid.myhost.org" /&gt;
+</screen>
+The url parameter should be a pseudo-URI formed with the URI scheme being
+the name of the provider to use to submit the coaster head job, and the
+hostname portion being the hostname to be used to execute the coaster
+head job. Note that this provider and hostname will be used for execution
+of a coaster head job, not for file transfer; so for example, a GRAM
+endpoint should be specified here rather than a GridFTP endpoint.
+</para>
+	</section>
+
 </article>
 



From noreply at svn.ci.uchicago.edu  Sat Oct 11 17:34:27 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sat, 11 Oct 2008 17:34:27 -0500 (CDT)
Subject: [Swift-commit] r2288 - log-processing/libexec
Message-ID: <20081011223427.D58AD2281BB@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-11 17:34:26 -0500 (Sat, 11 Oct 2008)
New Revision: 2288

Modified:
   log-processing/libexec/trail
   log-processing/libexec/trail.plot.template
Log:
make all states in trail plots start at 0,0 and shift everything up so the first event is numbered 1 not 0

Modified: log-processing/libexec/trail
===================================================================
--- log-processing/libexec/trail	2008-10-11 09:06:25 UTC (rev 2287)
+++ log-processing/libexec/trail	2008-10-11 22:34:26 UTC (rev 2288)
@@ -11,12 +11,13 @@
  grep -E "^[^ ]* *[^ ]* *${state}(\$| .*$)" $1.transitions > tmp-ss-$state
  cat tmp-ss-$state | sed 's/^\([^ ]*\) .*$/\1/' > tmp-ss2-$state
  sort -n < tmp-ss2-$state > tmp-ss0-$state
- I=0
+ I=1
  while read t rest; do
    echo $t $I
    I=$(( $I + 1 ))
  done < tmp-ss0-$state > tmp-ss9-$state
- ./normalise-event-start-time < tmp-ss9-$state > tmp-ss3-$state
+ echo 0 0 > tmp-ss3-$state
+ ./normalise-event-start-time < tmp-ss9-$state >> tmp-ss3-$state
  if [ "X$FIRSTSTATE" == "XNO" ]; then
    echo -n ", " >> tmp-trail.plot
  else

Modified: log-processing/libexec/trail.plot.template
===================================================================
--- log-processing/libexec/trail.plot.template	2008-10-11 09:06:25 UTC (rev 2287)
+++ log-processing/libexec/trail.plot.template	2008-10-11 22:34:26 UTC (rev 2288)
@@ -7,6 +7,7 @@
 set xlabel 'realtime / s'
 
 set xrange [0:MAXTIME]
+set yrange [0:]
 
 
 plot \



From noreply at svn.ci.uchicago.edu  Sat Oct 11 17:53:50 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sat, 11 Oct 2008 17:53:50 -0500 (CDT)
Subject: [Swift-commit] r2289 - in trunk/docs: . plot-tour
	plot-tour/pregenerated
Message-ID: <20081011225350.7039E2281B7@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-11 17:53:49 -0500 (Sat, 11 Oct 2008)
New Revision: 2289

Added:
   trunk/docs/plot-tour.xml
   trunk/docs/plot-tour/
   trunk/docs/plot-tour/execute.dot
   trunk/docs/plot-tour/execute2.dot
   trunk/docs/plot-tour/info.dot
   trunk/docs/plot-tour/logrelations.dot
   trunk/docs/plot-tour/pregenerated/
   trunk/docs/plot-tour/pregenerated/execute-dep.png
   trunk/docs/plot-tour/pregenerated/execute.png
Modified:
   trunk/docs/Makefile
Log:
beginnings of an introduction to log plots and what is going on inside Swift during a run

Modified: trunk/docs/Makefile
===================================================================
--- trunk/docs/Makefile	2008-10-11 22:34:26 UTC (rev 2288)
+++ trunk/docs/Makefile	2008-10-11 22:53:49 UTC (rev 2289)
@@ -2,7 +2,7 @@
 
 all: phps pdfs
 
-phps: userguide.php tutorial.php tutorial-live.php quickstartguide.php reallyquickstartguide.php languagespec.php languagespec-0.6.php log-processing.php
+phps: userguide.php tutorial.php tutorial-live.php quickstartguide.php reallyquickstartguide.php languagespec.php languagespec-0.6.php log-processing.php plot-tour.php
 
 pdfs: userguide.pdf tutorial.pdf tutorial-live.pdf quickstartguide.pdf reallyquickstartguide.pdf languagespec.pdf languagespec-0.6.pdf log-processing.pdf
 

Added: trunk/docs/plot-tour/execute.dot
===================================================================
--- trunk/docs/plot-tour/execute.dot	                        (rev 0)
+++ trunk/docs/plot-tour/execute.dot	2008-10-11 22:53:49 UTC (rev 2289)
@@ -0,0 +1,7 @@
+digraph EXECUTE {
+START -> END_SUCCESS;
+START -> END_FAILURE;
+START [shape=box,style=filled,color=green];
+END_SUCCESS [shape=box,style=filled,color=green];
+END_FAILURE [shape=box,style=filled,color=red];
+}

Added: trunk/docs/plot-tour/execute2.dot
===================================================================
--- trunk/docs/plot-tour/execute2.dot	                        (rev 0)
+++ trunk/docs/plot-tour/execute2.dot	2008-10-11 22:53:49 UTC (rev 2289)
@@ -0,0 +1,14 @@
+digraph EXECUTE2 {
+THREAD_ASSOCIATION -> JOB_START -> STAGING_OUT -> JOB_END;
+JOB_START -> JOB_CANCELLED;
+STAGING_OUT -> JOB_CANCELLED;
+JOB_START -> APPLICATION_EXCEPTION;
+STAGING_OUT -> APPLICATION_EXCEPTION;
+
+THREAD_ASSOCIATION [shape=box,style=filled,color=green];
+JOB_START [shape=box,style=filled,color=green];
+STAGING_OUT [shape=box,style=filled,color=green];
+JOB_END [shape=box,style=filled,color=green];
+JOB_CANCELLED [shape=box,style=filled,color=red];
+APPLICATION_EXCEPTION [shape=box,style=filled,color=red];
+}

Added: trunk/docs/plot-tour/info.dot
===================================================================
--- trunk/docs/plot-tour/info.dot	                        (rev 0)
+++ trunk/docs/plot-tour/info.dot	2008-10-11 22:53:49 UTC (rev 2289)
@@ -0,0 +1,16 @@
+digraph INFO {
+
+LOG_START -> CREATE_JOBDIR -> CREATE_INPUTDIR -> LINK_INPUTS -> EXECUTE -> EXECUTE_DONE -> COPYING_OUTPUTS -> RM_JOBDIR -> TOUCH_SUCCESS -> END;
+
+LOG_START [shape=box,style=filled,color=green];
+CREATE_JOBDIR [shape=box,style=filled,color=green];
+CREATE_INPUTDIR [shape=box,style=filled,color=green];
+LINK_INPUTS [shape=box,style=filled,color=green];
+EXECUTE [shape=box,style=filled,color=green];
+EXECUTE_DONE [shape=box,style=filled,color=green];
+COPYING_OUTPUTS [shape=box,style=filled,color=green];
+RM_JOBDIR [shape=box,style=filled,color=green];
+TOUCH_SUCCESS [shape=box,style=filled,color=green];
+END [shape=box,style=filled,color=green];
+
+}

Added: trunk/docs/plot-tour/logrelations.dot
===================================================================
--- trunk/docs/plot-tour/logrelations.dot	                        (rev 0)
+++ trunk/docs/plot-tour/logrelations.dot	2008-10-11 22:53:49 UTC (rev 2289)
@@ -0,0 +1,14 @@
+digraph EXECUTE {
+execute -> execute2;
+execute2 -> stagein -> karajan_FILE_TRANSFER;
+stagein -> karajan_FILE_OPERATION;
+execute2 -> karajan_JOB_SUBMISSION;
+execute2 -> stageout -> karajan_FILE_TRANSFER;
+stageout -> karajan_FILE_OPERATION;
+karajan_JOB_SUBMISSION -> info;
+subgraph cluster_KARAJAN {
+karajan_FILE_TRANSFER;
+karajan_FILE_OPERATION;
+karajan_JOB_SUBMISSION;
+}
+}

Added: trunk/docs/plot-tour/pregenerated/execute-dep.png
===================================================================
(Binary files differ)


Property changes on: trunk/docs/plot-tour/pregenerated/execute-dep.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Added: trunk/docs/plot-tour/pregenerated/execute.png
===================================================================
(Binary files differ)


Property changes on: trunk/docs/plot-tour/pregenerated/execute.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Added: trunk/docs/plot-tour.xml
===================================================================
--- trunk/docs/plot-tour.xml	                        (rev 0)
+++ trunk/docs/plot-tour.xml	2008-10-11 22:53:49 UTC (rev 2289)
@@ -0,0 +1,172 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook V4.2//EN" "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [] >
+
+<article>
+	<articleinfo revision="$LastChangedRevision$">
+		<title>Swift log plotting and the some internal mechanics of Swift</title>
+	</articleinfo>
+
+	<section id="overview">
+		<title>Overview</title>
+<para>
+This document attempts to explain some of the meaning of the Swift
+log-processing plots, giving an explanation of how some of Swift's
+execution mechanism works and of some of the terminology used.
+</para>
+	</section>
+
+<section id="execute"><title>'execute' - SwiftScript app {} block invocations</title>
+
+<para>
+When a SwiftScript program invokes a application procedure (one with an
+app {} block), an 'execute' appears in the log file in START state. When
+all attempts at execution have finished (either successfully or unsuccessfully)
+then the execute will go into END_SUCCESS or END_FAILURE state. A workflow
+is successful if and only if all invocations end in END_SUCCESS.
+</para>
+
+<para>
+The execute states represent progress
+through the karajan procedure defined in
+<filename>libexec/execute-default.k</filename>.
+</para>
+<para>State changes for execute logs are defined by karajan log calls throughout
+this file.
+</para>
+<inlinemediaobject><imageobject><imagedata fileref="execute.png"></imagedata></imageobject></inlinemediaobject>
+
+<para>An execute consists of multiple attempts to perform
+<link linkend="execute2">execute2</link>s, with retries and replication
+as appropriate. Retries and replication are not exposed through the states
+of 'execute's.
+</para>
+<para>
+Executes are uniquely identified within a run by their karajan thread ID,
+which is present in the log files as the thread= parameter on execute
+log messsages.
+</para>
+
+<para>
+Here is a simple SwiftScript program which runs a foreach loop (<filename>few.swift</filename>):
+<programlisting>
+p() { 
+    app {
+        sleep "10s";
+    }
+}
+
+foreach i in [1:8] {
+    p();
+}
+</programlisting>
+
+</para>
+
+<para>
+Using the <command>swift-plot-log</command> from the log processing module,
+this graph gets generated to summarise execute state transitions:
+</para>
+<para>
+<inlinemediaobject><imageobject><imagedata fileref="plot-tour/pregenerated/execute.png"></imagedata></imageobject></inlinemediaobject>
+</para>
+<para>
+In this graph, the forloop calls p() eight times. Because there are no
+dependencies between those eight invocations, they are all invoked at the same
+time, around 1s into the run. This is show on the graph by the JOB_START line
+going from zero up to eight at around x=1s. As time passes, the sleep jobs
+complete, and as they do so the number of jobs in END_SUCCESS state increases.
+When all eight jobs are in END_SUCCESS state, the run is over.
+</para>
+<para>Here is a program with some data dependencies between invocations (<filename>dep.swift</filename>):
+
+<programlisting>
+$ cat dep.swift 
+type file;
+
+p(file f) { 
+    app {
+        sleep "10s";
+    }
+}
+
+(file o) q() {
+    app {
+        touch @o;
+    }
+}
+
+file intermediate = q();
+p(intermediate);
+</programlisting>
+
+</para>
+
+<para>
+Here is a plot of the execute states for this program:
+</para>
+<para><inlinemediaobject><imageobject><imagedata fileref="plot-tour/pregenerated/execute-dep.png"></imagedata></imageobject></inlinemediaobject>
+</para>
+<para>
+In this run, one invocation starts (q()) fairly quickly, 
+but the other invocation (of p()) does not - instead, it does not start until
+approximately the time that the q() invocation has reached END_SUCCESS. 
+</para>
+</section>
+<section id="execute2"><title>execute2 - one attempt at running an execute</title>
+<para>
+An execute2 is one attempt to execute an app procedure. execute2s are invoked
+by <link linkend="execute">execute</link>, once for each retry or replication
+attempt.
+</para>
+<para>The states of an execute2 represent progress through the execute2 karajan
+procedure defined in <filename>libexec/vdl-int.k</filename>
+</para>
+<inlinemediaobject><imageobject><imagedata fileref="execute2.png"></imagedata></imageobject></inlinemediaobject>
+<para>
+Before an execute2 makes its first state log entry, it chooses a site to run on.
+Then at the start of file stage-in, the execute2 goes into THREAD_ASSOCIATION
+state. Once stagein is completed, the JOB_START state is entered, indicating
+that execution of the job executable will now be attempted. Following that,
+STAGING_OUT indicates that the output files are being staged out. If everything
+is completed successfully, the job will enter JOB_END state.
+</para>
+<para>There are two exceptions to the above sequence: JOB_CANCELLED indicates that
+the replication mechanism has cancelled this job because a different execute2
+began actual execution on a site for the same execute. APPLICATION_EXCEPTION
+indicates that there was an error somewhere in the attempt to stage in,
+actually execute or stage out. If a job goes into APPLICATION_EXCEPTION state
+then it will generally be retried (up to a certain number of times defined
+by the "execution.retries" parameter) by the containing <link linkend="execute">execute</link>.
+</para>
+</section>
+
+<section id="info"><title>wrapper info logs</title>
+<para>
+When a job runs, it is wrapped by a Swift shell script on the remote site that
+prepares the job environment, creating a temporary directory and moving
+input and output files around. Each wrapper invocation corresponds to a single
+application execution. For each invocation of the wrapper, a log file is created.
+Sometimes that log file is moved back to the submission side (when there is
+an error during execution, or when the setting 
+<ulink url="http://www.ci.uchicago.edu/swift/guides/userguide.php#engineconfiguration">wrapper.always.transfer</ulink>=true
+is set) and placed in a <filename>*.d/</filename> directory corresponding in
+name to the main log file.
+</para>
+
+<inlinemediaobject><imageobject><imagedata fileref="info.png"></imagedata></imageobject></inlinemediaobject>
+<para>The states of the info logs represent progress through the wrapper
+script, <filename>libexec/wrapper.sh</filename>.
+</para>
+
+</section>
+
+<section><title>Relation of logged entities to each other</title>
+<para>Here is a simple diagram of how some of the above log channels along
+with other pieces fit together:</para>
+<inlinemediaobject><imageobject><imagedata fileref="logrelations.png"></imagedata></imageobject></inlinemediaobject>
+
+</section>
+
+</article>
+



From noreply at svn.ci.uchicago.edu  Mon Oct 13 00:07:36 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 13 Oct 2008 00:07:36 -0500 (CDT)
Subject: [Swift-commit] r2290 - in trunk/docs: . plot-tour/pregenerated
Message-ID: <20081013050736.F3EA82281B7@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-13 00:07:36 -0500 (Mon, 13 Oct 2008)
New Revision: 2290

Added:
   trunk/docs/plot-tour/pregenerated/execute-many-dep.png
   trunk/docs/plot-tour/pregenerated/few2.png
Modified:
   trunk/docs/plot-tour.xml
Log:
an example for the execute section showing a multistage foreach loop run

Added: trunk/docs/plot-tour/pregenerated/execute-many-dep.png
===================================================================
(Binary files differ)


Property changes on: trunk/docs/plot-tour/pregenerated/execute-many-dep.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Added: trunk/docs/plot-tour/pregenerated/few2.png
===================================================================
(Binary files differ)


Property changes on: trunk/docs/plot-tour/pregenerated/few2.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Modified: trunk/docs/plot-tour.xml
===================================================================
--- trunk/docs/plot-tour.xml	2008-10-11 22:53:49 UTC (rev 2289)
+++ trunk/docs/plot-tour.xml	2008-10-13 05:07:36 UTC (rev 2290)
@@ -112,6 +112,56 @@
 but the other invocation (of p()) does not - instead, it does not start until
 approximately the time that the q() invocation has reached END_SUCCESS. 
 </para>
+
+<para>
+Finally in this section on 'execute', here is a demonstration of how the above
+two patterns fit together in one program (<filename>few2.swift</filename>:
+<programlisting>
+type file;
+
+(file o) p(file i) { 
+    app {
+        sleepcopy @i @o;
+    }
+}
+
+file input &lt;"input"&gt;;
+file output[];
+
+foreach i in [1:8] {
+    file intermediate;
+    intermediate = p(input);
+    output[i] = p(intermediate);
+}
+</programlisting>
+</para>
+
+
+<para>
+In total the program has 16 invocations of p(), dependent on each other in
+pairs. The dependencies can be plotted like this:
+
+<screen>
+$ <userinput>swift -pgraph few2.dot few2.swift</userinput>
+$ dot -Tpng -o few2.png few2.dot 
+</screen>
+
+yielding this graph:
+</para>
+
+<para><inlinemediaobject><imageobject><imagedata fileref="plot-tour/pregenerated/few2.png"></imagedata></imageobject></inlinemediaobject> </para>
+
+<para>
+When this program is run, the first row of 8 invocations can all start at the
+beginning of the program, because they have no dependencies (aside from on
+the input file). This can be seen around t=4 when the start line jumps up to 8.
+The other 8 invocations can only begin when the invocations they are dependent
+on have finished. This can be seen in the graph - every time one of the first
+invocations reaches END_SUCCESS, a new invocation enters START.
+</para>
+
+<para><inlinemediaobject><imageobject><imagedata fileref="plot-tour/pregenerated/execute-many-dep.png"></imagedata></imageobject></inlinemediaobject> </para>
+
 </section>
 <section id="execute2"><title>execute2 - one attempt at running an execute</title>
 <para>



From noreply at svn.ci.uchicago.edu  Mon Oct 13 01:15:01 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 13 Oct 2008 01:15:01 -0500 (CDT)
Subject: [Swift-commit] r2291 - in trunk/docs: . plot-tour/pregenerated
Message-ID: <20081013061501.79FF3228197@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-13 01:15:00 -0500 (Mon, 13 Oct 2008)
New Revision: 2291

Added:
   trunk/docs/plot-tour/pregenerated/execute2.png
Modified:
   trunk/docs/plot-tour.xml
Log:
graph for execute2

Added: trunk/docs/plot-tour/pregenerated/execute2.png
===================================================================
(Binary files differ)


Property changes on: trunk/docs/plot-tour/pregenerated/execute2.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Modified: trunk/docs/plot-tour.xml
===================================================================
--- trunk/docs/plot-tour.xml	2008-10-13 05:07:36 UTC (rev 2290)
+++ trunk/docs/plot-tour.xml	2008-10-13 06:15:00 UTC (rev 2291)
@@ -189,6 +189,54 @@
 then it will generally be retried (up to a certain number of times defined
 by the "execution.retries" parameter) by the containing <link linkend="execute">execute</link>.
 </para>
+
+<para>
+In this example, we use a large input file to slow down file staging so that
+it is visible on an execute2 graph (<filename>big-files.swift</filename>):
+<programlisting>
+type file;  
+  
+(file o) p(file i) {   
+    app {  
+        sleepcopy @i @o;  
+    }  
+}  
+  
+file input &lt;"biginput"&gt;;  
+file output[];  
+  
+foreach i in [1:8] {  
+    output[i] = p(input);  
+}  
+</programlisting>
+</para>
+
+<para>
+<inlinemediaobject><imageobject><imagedata fileref="plot-tour/pregenerated/execute2.png"></imagedata></imageobject></inlinemediaobject></para>
+
+<para>
+There is an initial large input file that must be staged in. This causes the first
+jobs to be in stagein state for a period of time (the space between the
+ASSOCIATED and JOB_START lines at the lower left corner of the graph). All
+invocations share a single input file, so it is only staged in once and
+shared between all subsequent invocations - once the file has staged in at the
+start, there is no space later on between the ASSOCIATED and JOB_START lines
+because of this.
+</para>
+<para>
+Conversely, each invocation generates a large output file without there being
+any sharing. Each of those output files must be staged back to the submit
+side, which in this application takes some time. This can be seen by the large
+amount of space between the STAGING_OUT and JOB_END lines.
+</para>
+<para>
+The remaining large space on the graph is between the JOB_START and STAGING_OUT
+lines. This represents the time taken to queue and execute the application
+executable (and surrounding Swift worker-side wrapper, which can sometimes
+have non-negligable execution times - this can be seen in the
+<link linkend="info">info section</link>).
+</para>
+
 </section>
 
 <section id="info"><title>wrapper info logs</title>



From noreply at svn.ci.uchicago.edu  Mon Oct 13 15:08:21 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 13 Oct 2008 15:08:21 -0500 (CDT)
Subject: [Swift-commit] r2292 - trunk/src/org/griphyn/vdl/type
Message-ID: <20081013200821.BB6CD2281A3@www.ci.uchicago.edu>

Author: hategan
Date: 2008-10-13 15:08:20 -0500 (Mon, 13 Oct 2008)
New Revision: 2292

Modified:
   trunk/src/org/griphyn/vdl/type/Types.java
Log:
fixed cme; getType may add types

Modified: trunk/src/org/griphyn/vdl/type/Types.java
===================================================================
--- trunk/src/org/griphyn/vdl/type/Types.java	2008-10-13 06:15:00 UTC (rev 2291)
+++ trunk/src/org/griphyn/vdl/type/Types.java	2008-10-13 20:08:20 UTC (rev 2292)
@@ -68,7 +68,8 @@
 	}
 
 	public synchronized static void resolveTypes() throws NoSuchTypeException {
-		Iterator i = types.entrySet().iterator();
+		Map typesCopy = new HashMap(types);
+		Iterator i = typesCopy.entrySet().iterator();
 		while (i.hasNext()) {
 			Map.Entry e = (Map.Entry) i.next();
 			Type type = (Type) e.getValue();



From noreply at svn.ci.uchicago.edu  Mon Oct 13 21:11:16 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 13 Oct 2008 21:11:16 -0500 (CDT)
Subject: [Swift-commit] r2293 - log-processing/libexec
Message-ID: <20081014021116.A910522819B@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-13 21:11:15 -0500 (Mon, 13 Oct 2008)
New Revision: 2293

Modified:
   log-processing/libexec/info-and-karajan-actives
Log:
ignore gnuplot exit code so that build continues even without wrapper info files present

Modified: log-processing/libexec/info-and-karajan-actives
===================================================================
--- log-processing/libexec/info-and-karajan-actives	2008-10-13 20:08:20 UTC (rev 2292)
+++ log-processing/libexec/info-and-karajan-actives	2008-10-14 02:11:15 UTC (rev 2293)
@@ -51,4 +51,5 @@
 
 gnuplot info-and-karajan-actives.2.plot
 
-
+# force successful exit rather than passing along the gnuplot exit code
+exit 0



From noreply at svn.ci.uchicago.edu  Tue Oct 14 01:43:32 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 14 Oct 2008 01:43:32 -0500 (CDT)
Subject: [Swift-commit] r2294 - in trunk/docs: . plot-tour/pregenerated
Message-ID: <20081014064332.7CBC222819B@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-14 01:43:31 -0500 (Tue, 14 Oct 2008)
New Revision: 2294

Added:
   trunk/docs/plot-tour/pregenerated/info.png
Modified:
   trunk/docs/plot-tour.xml
Log:
some notes about info/wrapper logs

Added: trunk/docs/plot-tour/pregenerated/info.png
===================================================================
(Binary files differ)


Property changes on: trunk/docs/plot-tour/pregenerated/info.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Modified: trunk/docs/plot-tour.xml
===================================================================
--- trunk/docs/plot-tour.xml	2008-10-14 02:11:15 UTC (rev 2293)
+++ trunk/docs/plot-tour.xml	2008-10-14 06:43:31 UTC (rev 2294)
@@ -257,6 +257,47 @@
 script, <filename>libexec/wrapper.sh</filename>.
 </para>
 
+<para>
+For the same run of <filename>big-files.swift</filename> as shown in the
+<link linkend="execute2">execute2 section</link>, here is a plot of states
+in wrapper info log files:
+</para>
+
+<para>
+<inlinemediaobject><imageobject><imagedata fileref="plot-tour/pregenerated/info.png"></imagedata></imageobject></inlinemediaobject></para>
+
+<para>
+The trace lines on this graph fit entirely within the space between JOB_START 
+and STAGING_OUT on the corresponding execute2 graph, because the Swift worker node
+wrapper script does not run until the submit side of Swift has submitted a
+job for execution and that job has begun running.
+</para>
+
+<para>
+Many of the lines on this plot are very close together, because many of the
+operations take minimal time. The main space between lines is between
+EXECUTE and EXECUTE_DONE, where the actual application executable is executing;
+and between COPYING_OUTPUTS and RM_JOBDIR, where the large output files are
+copied from a job specific working directory to the site-specific shared
+directory. It is quite hard to distinguish on the graph where overlapping
+lines are plotted together.
+</para>
+
+<para>
+Note also that minimal time is spent copying input files into the job-specific
+directory in the wrapper script; that is because in this run, the wrapper
+script is using the default behaviour of making symbolic links in the job-specific
+directory; symbolic links are usually cheap to create compared to copying file
+content. However, if the <ulink url="http://www.ci.uchicago.edu/swift/guides/userguide.php#envvars">SWIFT_JOBDIR_PATH</ulink> parameter is set, then Swift will
+copy the input file to the specified job directory instead of linking. This
+will generally result in much more time being spent preparing the job directory
+in the Swift wrapper, but in certain circumstances this time is overwhelmingly
+offset by increased performance of the actual application executable (so on
+this chart, this would be seen as an increased job directory preparation time,
+but a reduced-by-more application executable time).
+</para>
+
+
 </section>
 
 <section><title>Relation of logged entities to each other</title>



From noreply at svn.ci.uchicago.edu  Tue Oct 14 01:54:09 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 14 Oct 2008 01:54:09 -0500 (CDT)
Subject: [Swift-commit] r2295 - log-processing/libexec
Message-ID: <20081014065409.F2499228197@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-14 01:54:09 -0500 (Tue, 14 Oct 2008)
New Revision: 2295

Modified:
   log-processing/libexec/trail.plot.template
Log:
nicer y-axis label label

Modified: log-processing/libexec/trail.plot.template
===================================================================
--- log-processing/libexec/trail.plot.template	2008-10-14 06:43:31 UTC (rev 2294)
+++ log-processing/libexec/trail.plot.template	2008-10-14 06:54:09 UTC (rev 2295)
@@ -3,7 +3,7 @@
 set output 'tmp-trail.png'
 
 set key outside below
-set ylabel 'total events'
+set ylabel 'count of events that have reached state'
 set xlabel 'realtime / s'
 
 set xrange [0:MAXTIME]



From noreply at svn.ci.uchicago.edu  Tue Oct 14 02:00:45 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 14 Oct 2008 02:00:45 -0500 (CDT)
Subject: [Swift-commit] r2296 - log-processing/libexec
Message-ID: <20081014070045.95AD122819B@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-14 02:00:45 -0500 (Tue, 14 Oct 2008)
New Revision: 2296

Modified:
   log-processing/libexec/makefile
Log:
clean HTMLs

Modified: log-processing/libexec/makefile
===================================================================
--- log-processing/libexec/makefile	2008-10-14 06:54:09 UTC (rev 2295)
+++ log-processing/libexec/makefile	2008-10-14 07:00:45 UTC (rev 2296)
@@ -16,7 +16,7 @@
 	cp *.event *.html *.png report-$(SDL)
 
 clean:
-	rm -f start-times.data kickstart-times.data start-time.tmp end-time.tmp threads.list tasks.list log *.data *.shifted *.png *.event *.coloured-event *.total *.tmp *.transitions *.last karatasks-type-counts.txt index.html *.lastsummary execstages.plot total.plot colour.plot jobs-sites.table jobs.retrycount.summary kickstart.stats execution-counts.txt site-duration.txt jobs.retrycount sp.plot karatasks.coloured-sorted-event *.cedps *.stats t.inf *.seenstates tmp-* clusterstats  trname-summary sites-list.data.nm info-md5sums pse2d-tmp.eip
+	rm -f start-times.data kickstart-times.data start-time.tmp end-time.tmp threads.list tasks.list log *.data *.shifted *.png *.event *.coloured-event *.total *.tmp *.transitions *.last karatasks-type-counts.txt index.html *.lastsummary execstages.plot total.plot colour.plot jobs-sites.table jobs.retrycount.summary kickstart.stats execution-counts.txt site-duration.txt jobs.retrycount sp.plot karatasks.coloured-sorted-event *.cedps *.stats t.inf *.seenstates tmp-* clusterstats  trname-summary sites-list.data.nm info-md5sums pse2d-tmp.eip karajan.html falkon.html execute2.html info.html execute.html kickstart.html scheduler.html assorted.html
 
 t.inf: $(LOG)
 	./compute-t-inf > t.inf < $(LOG)



From noreply at svn.ci.uchicago.edu  Tue Oct 14 11:34:57 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 14 Oct 2008 11:34:57 -0500 (CDT)
Subject: [Swift-commit] r2297 - trunk/tests/language-behaviour
Message-ID: <20081014163457.D0EEC228135@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-14 11:34:56 -0500 (Tue, 14 Oct 2008)
New Revision: 2297

Added:
   trunk/tests/language-behaviour/regress-r2292.swift
Log:
a regression test for r2292

Added: trunk/tests/language-behaviour/regress-r2292.swift
===================================================================
--- trunk/tests/language-behaviour/regress-r2292.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/regress-r2292.swift	2008-10-14 16:34:56 UTC (rev 2297)
@@ -0,0 +1 @@
+type row { int column[]; }



From noreply at svn.ci.uchicago.edu  Tue Oct 14 13:32:49 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 14 Oct 2008 13:32:49 -0500 (CDT)
Subject: [Swift-commit] r2298 - in trunk/src/org/griphyn/vdl: engine
	karajan/lib/swiftscript
Message-ID: <20081014183249.E64C1228135@www.ci.uchicago.edu>

Author: hategan
Date: 2008-10-14 13:32:48 -0500 (Tue, 14 Oct 2008)
New Revision: 2298

Added:
   trunk/src/org/griphyn/vdl/karajan/lib/swiftscript/ReadData2.java
Modified:
   trunk/src/org/griphyn/vdl/engine/ProcedureSignature.java
Log:
added readData2

Modified: trunk/src/org/griphyn/vdl/engine/ProcedureSignature.java
===================================================================
--- trunk/src/org/griphyn/vdl/engine/ProcedureSignature.java	2008-10-14 16:34:56 UTC (rev 2297)
+++ trunk/src/org/griphyn/vdl/engine/ProcedureSignature.java	2008-10-14 18:32:48 UTC (rev 2298)
@@ -104,6 +104,13 @@
 		readData.addOutputArg(rdOutputArg);		
 		proceduresMap.put("readData", readData);
 		
+		ProcedureSignature readData2 = new ProcedureSignature("readData2");
+        FormalArgumentSignature rd2InputArg = new FormalArgumentSignature(true);
+        readData2.addInputArg(rd2InputArg);
+        FormalArgumentSignature rd2OutputArg = new FormalArgumentSignature(true);
+        readData2.addOutputArg(rd2OutputArg);     
+        proceduresMap.put("readData2", readData2);
+		
 		ProcedureSignature print = new ProcedureSignature("print");
 		print.setAnyNumOfInputArgs();
 		proceduresMap.put("print", print);

Added: trunk/src/org/griphyn/vdl/karajan/lib/swiftscript/ReadData2.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/swiftscript/ReadData2.java	                        (rev 0)
+++ trunk/src/org/griphyn/vdl/karajan/lib/swiftscript/ReadData2.java	2008-10-14 18:32:48 UTC (rev 2298)
@@ -0,0 +1,122 @@
+/*
+ * Created on Oct 8, 2007
+ */
+package org.griphyn.vdl.karajan.lib.swiftscript;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileReader;
+import java.io.IOException;
+
+import org.apache.log4j.Logger;
+import org.globus.cog.karajan.arguments.Arg;
+import org.globus.cog.karajan.stack.VariableStack;
+import org.globus.cog.karajan.workflow.ExecutionException;
+import org.griphyn.vdl.karajan.lib.VDLFunction;
+import org.griphyn.vdl.mapping.AbsFile;
+import org.griphyn.vdl.mapping.DSHandle;
+import org.griphyn.vdl.mapping.HandleOpenException;
+import org.griphyn.vdl.mapping.Path;
+import org.griphyn.vdl.mapping.PhysicalFormat;
+import org.griphyn.vdl.type.Types;
+
+public class ReadData2 extends VDLFunction {
+	public static final Logger logger = Logger.getLogger(ReadData2.class);
+
+	public static final Arg DEST = new Arg.Positional("dest");
+	public static final Arg SRC = new Arg.Positional("src");
+	public static boolean warning;
+
+	static {
+		setArguments(ReadData2.class, new Arg[] { DEST, SRC });
+	}
+
+	protected Object function(VariableStack stack) throws ExecutionException, HandleOpenException {
+		DSHandle dest = (DSHandle) DEST.getValue(stack);
+		DSHandle src = (DSHandle) SRC.getValue(stack);
+		waitFor(stack, src);
+		if (src.getType().equals(Types.STRING)) {
+			readData(dest, (String) src.getValue());
+		}
+		else {
+			PhysicalFormat pf = src.getMapper().map(Path.EMPTY_PATH);
+			if (pf instanceof AbsFile) {
+				AbsFile af = (AbsFile) pf;
+				if (!af.getProtocol().equalsIgnoreCase("file")) {
+					throw new ExecutionException("readData only supports local files");
+				}
+				readData(dest, af.getPath());
+			}
+			else {
+				throw new ExecutionException("readData only supports reading from files");
+			}
+		}
+		return null;
+	}
+
+	private void readData(DSHandle dest, String path) throws ExecutionException {
+		File f = new File(path);
+		try {
+			BufferedReader br = new BufferedReader(new FileReader(f));
+			try {
+				readLines(dest, br, path);
+			}
+			finally {
+				try {
+					br.close();
+				}
+				catch (IOException e) {
+					e.printStackTrace();
+				}
+			}
+		}
+		catch (Exception e) {
+			throw new ExecutionException(e);
+		}
+	}
+
+	private void readLines(DSHandle dest, BufferedReader br, String path)
+			throws ExecutionException, IOException {
+		int count = 1;
+		String line = br.readLine();
+		while (line != null) {
+			line = line.trim();
+			if (!line.startsWith("#") && !line.equals("")) {
+				try {
+					String[] sp = line.split("=", 2);
+					setValue(dest.getField(Path.parse(sp[0].trim())), sp[1].trim());
+				}
+				catch (Exception e) {
+					throw new ExecutionException(e.getMessage() + " in " + path + ", line " + count
+							+ ": " + line, e);
+				}
+			}
+			line = br.readLine();
+			count++;
+		}
+	}
+
+	private void setValue(DSHandle dest, String s) throws ExecutionException {
+		try {
+			if (dest.getType().equals(Types.INT)) {
+				dest.setValue(new Double(Integer.parseInt(s.trim())));
+			}
+			else if (dest.getType().equals(Types.FLOAT)) {
+				dest.setValue(new Double(s.trim()));
+			}
+			else if (dest.getType().equals(Types.BOOLEAN)) {
+				dest.setValue(new Boolean(s.trim()));
+			}
+			else if (dest.getType().equals(Types.STRING)) {
+				dest.setValue(s);
+			}
+			else {
+				throw new ExecutionException("Don't know how to read type " + dest.getType()
+						+ " for path " + dest.getPathFromRoot());
+			}
+		}
+		catch (NumberFormatException e) {
+			throw new ExecutionException("Could not convert value to number: " + s);
+		}
+	}
+}



From noreply at svn.ci.uchicago.edu  Tue Oct 14 13:34:17 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 14 Oct 2008 13:34:17 -0500 (CDT)
Subject: [Swift-commit] r2299 - trunk/libexec
Message-ID: <20081014183417.8FD6622819B@www.ci.uchicago.edu>

Author: hategan
Date: 2008-10-14 13:34:17 -0500 (Tue, 14 Oct 2008)
New Revision: 2299

Modified:
   trunk/libexec/vdl-lib.xml
Log:
added readData2

Modified: trunk/libexec/vdl-lib.xml
===================================================================
--- trunk/libexec/vdl-lib.xml	2008-10-14 18:32:48 UTC (rev 2298)
+++ trunk/libexec/vdl-lib.xml	2008-10-14 18:34:17 UTC (rev 2299)
@@ -4,6 +4,7 @@
 	<!-- string functions library -->
 	<export name="extractint"><elementDef classname="org.griphyn.vdl.karajan.lib.swiftscript.ExtractInt"/></export>
 	<export name="readdata"><elementDef classname="org.griphyn.vdl.karajan.lib.swiftscript.ReadData"/></export>
+	<export name="readdata2"><elementDef classname="org.griphyn.vdl.karajan.lib.swiftscript.ReadData2"/></export>
 	<export name="strcat"><elementDef classname="org.griphyn.vdl.karajan.lib.swiftscript.Misc"/></export>
 	<export name="strcut"><elementDef classname="org.griphyn.vdl.karajan.lib.swiftscript.Misc"/></export>
 	<export name="regexp"><elementDef classname="org.griphyn.vdl.karajan.lib.swiftscript.Misc"/></export>



From noreply at svn.ci.uchicago.edu  Wed Oct 15 03:53:47 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 15 Oct 2008 03:53:47 -0500 (CDT)
Subject: [Swift-commit] r2300 - trunk/src/org/griphyn/vdl/engine
Message-ID: <20081015085347.C1ED8228197@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-15 03:53:47 -0500 (Wed, 15 Oct 2008)
New Revision: 2300

Modified:
   trunk/src/org/griphyn/vdl/engine/ProcedureSignature.java
Log:
whitespace

Modified: trunk/src/org/griphyn/vdl/engine/ProcedureSignature.java
===================================================================
--- trunk/src/org/griphyn/vdl/engine/ProcedureSignature.java	2008-10-14 18:34:17 UTC (rev 2299)
+++ trunk/src/org/griphyn/vdl/engine/ProcedureSignature.java	2008-10-15 08:53:47 UTC (rev 2300)
@@ -105,11 +105,11 @@
 		proceduresMap.put("readData", readData);
 		
 		ProcedureSignature readData2 = new ProcedureSignature("readData2");
-        FormalArgumentSignature rd2InputArg = new FormalArgumentSignature(true);
-        readData2.addInputArg(rd2InputArg);
-        FormalArgumentSignature rd2OutputArg = new FormalArgumentSignature(true);
-        readData2.addOutputArg(rd2OutputArg);     
-        proceduresMap.put("readData2", readData2);
+		FormalArgumentSignature rd2InputArg = new FormalArgumentSignature(true);
+		readData2.addInputArg(rd2InputArg);
+		FormalArgumentSignature rd2OutputArg = new FormalArgumentSignature(true);
+		readData2.addOutputArg(rd2OutputArg);     
+		proceduresMap.put("readData2", readData2);
 		
 		ProcedureSignature print = new ProcedureSignature("print");
 		print.setAnyNumOfInputArgs();



From noreply at svn.ci.uchicago.edu  Wed Oct 15 05:28:14 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 15 Oct 2008 05:28:14 -0500 (CDT)
Subject: [Swift-commit] r2301 - trunk/tests/language-behaviour
Message-ID: <20081015102814.6824822819B@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-15 05:28:13 -0500 (Wed, 15 Oct 2008)
New Revision: 2301

Added:
   trunk/tests/language-behaviour/readData2.in
   trunk/tests/language-behaviour/readData2.out.expected
   trunk/tests/language-behaviour/readData2.swift
Log:
readData2 test

Added: trunk/tests/language-behaviour/readData2.in
===================================================================
--- trunk/tests/language-behaviour/readData2.in	                        (rev 0)
+++ trunk/tests/language-behaviour/readData2.in	2008-10-15 10:28:13 UTC (rev 2301)
@@ -0,0 +1,6 @@
+rows[0].columns[0] = 0                                                          
+rows[0].columns[1] = 2                                                          
+rows[0].columns[2] = 4                                                          
+rows[1].columns[0] = 1                                                          
+rows[1].columns[1] = 3                                                          
+rows[1].columns[2] = 5 

Added: trunk/tests/language-behaviour/readData2.out.expected
===================================================================
--- trunk/tests/language-behaviour/readData2.out.expected	                        (rev 0)
+++ trunk/tests/language-behaviour/readData2.out.expected	2008-10-15 10:28:13 UTC (rev 2301)
@@ -0,0 +1 @@
+15

Added: trunk/tests/language-behaviour/readData2.swift
===================================================================
--- trunk/tests/language-behaviour/readData2.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/readData2.swift	2008-10-15 10:28:13 UTC (rev 2301)
@@ -0,0 +1,33 @@
+type vector {  
+   int columns[];   
+}    
+
+type matrix {  
+   vector rows[];   
+}    
+
+type file;
+
+(file o) echo(int s) {
+  app {
+    echo s stdout=@o;
+  }
+}
+
+
+matrix m; 
+
+m = readData2("readData2.in"); 
+
+int s;
+
+s = m.rows[0].columns[0] +
+        m.rows[0].columns[1] +
+        m.rows[0].columns[2] +
+        m.rows[1].columns[0] +
+        m.rows[1].columns[1] +
+        m.rows[1].columns[2] ; 
+
+file out <"readData2.out">;
+out = echo(s);
+



From noreply at svn.ci.uchicago.edu  Wed Oct 15 17:23:13 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 15 Oct 2008 17:23:13 -0500 (CDT)
Subject: [Swift-commit] r2302 - trunk/docs
Message-ID: <20081015222313.1F390228135@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-15 17:23:12 -0500 (Wed, 15 Oct 2008)
New Revision: 2302

Modified:
   trunk/docs/userguide.xml
Log:
readdata2 userguide

Modified: trunk/docs/userguide.xml
===================================================================
--- trunk/docs/userguide.xml	2008-10-15 10:28:13 UTC (rev 2301)
+++ trunk/docs/userguide.xml	2008-10-15 22:23:12 UTC (rev 2302)
@@ -2220,6 +2220,43 @@
 This procedure is new in 0.4.
 			</para>
 		</section>
+		<section id="procedure.readdata2"><title>readdata2</title>
+			<para>
+readdata2 will read data from a specified file, like readdata, but using
+a different file format more closely related to that used by the
+ext mapper.
+			</para>
+			<para>
+Input files should list, one per line, a path into a Swift structure, and
+the value for that position in the structure:
+				<screen>
+rows[0].columns[0] = 0                                                          
+rows[0].columns[1] = 2                                                          
+rows[0].columns[2] = 4                                                          
+rows[1].columns[0] = 1                                                          
+rows[1].columns[1] = 3                                                          
+rows[1].columns[2] = 5 
+				</screen>
+which can be read into a structure defined like this:
+				<programlisting>
+type vector {                                                                   
+        int columns[];                                                          
+}                                                                               
+                                                                                
+type matrix {                                                                   
+        vector rows[];                                                          
+}                                                                               
+                                                                                
+matrix m;                                                                       
+                                                                                
+m = readData2("readData2.in");    
+				</programlisting>
+			</para>
+			
+			<para>
+This procedure is new in 0.7.
+			</para>
+		</section>
 		<section id="procedure.print"><title>print</title>
 			<para>
 Deprecated - use <link linkend="procedure.trace">trace</link> instead.



From noreply at svn.ci.uchicago.edu  Fri Oct 17 14:31:25 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 17 Oct 2008 14:31:25 -0500 (CDT)
Subject: [Swift-commit] r2303 - SwiftApps
Message-ID: <20081017193125.6B3132281A5@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-17 14:31:24 -0500 (Fri, 17 Oct 2008)
New Revision: 2303

Removed:
   SwiftApps/adem/
Log:
Deleting the old version



From noreply at svn.ci.uchicago.edu  Fri Oct 17 14:32:14 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 17 Oct 2008 14:32:14 -0500 (CDT)
Subject: [Swift-commit] r2304 - SwiftApps
Message-ID: <20081017193214.4751D2281A5@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-17 14:32:13 -0500 (Fri, 17 Oct 2008)
New Revision: 2304

Added:
   SwiftApps/adem-osg/
Log:
making a new dir for osg version



From noreply at svn.ci.uchicago.edu  Mon Oct 20 21:15:57 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 20 Oct 2008 21:15:57 -0500 (CDT)
Subject: [Swift-commit] r2307 - SwiftApps/adem-osg/bin
Message-ID: <20081021021557.7BFE422814F@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-20 21:15:56 -0500 (Mon, 20 Oct 2008)
New Revision: 2307

Added:
   SwiftApps/adem-osg/bin/auto-update-sites
Log:
New script import

Added: SwiftApps/adem-osg/bin/auto-update-sites
===================================================================
--- SwiftApps/adem-osg/bin/auto-update-sites	                        (rev 0)
+++ SwiftApps/adem-osg/bin/auto-update-sites	2008-10-21 02:15:56 UTC (rev 2307)
@@ -0,0 +1,80 @@
+# update if have new sites
+#echo "hello"
+DIR="$ADEM_HOME"
+USER=`whoami`
+GRID=$1
+VO=$2
+SFILE=$3
+ 
+for SITE in $(cat $SFILE)
+ do
+  newsite=`/bin/grep $SITE $DIR/bin/.osg-sites.cache`
+  if [ "X$newsite" == "X" ]; then
+  {
+  #get $APP for new sites
+  ID=`/bin/grep $SITE $DIR/tmp/$GRID-sitelist.txt |awk -F, '{print $1}'`
+  /usr/bin/wget -O $DIR/tmp/$ID-site.txt "http://vors.grid.iu.edu/cgi-bin/tindex.cgi?grid=$GRID&VO=$VO&dtype=0&res=$ID" 1>/dev/null 2>/dev/null
+  APP=`/bin/grep app_loc $DIR/tmp/$ID-site.txt |awk -F= '{print $2}'`
+  DATA=`/bin/grep data_loc $DIR/tmp/$ID-site.txt |awk -F= '{print $2}'`
+  LRM=`/bin/grep exec_jm $DIR/tmp/$ID-site.txt |awk -F/  '{print $2}' |cut -d- -f2`
+  NAME=`/bin/grep shortname $DIR/tmp/$ID-site.txt |awk -F= '{print $2}'` 
+
+  #get $Pacman for new sites
+  PacB=`globus-job-run $SITE  /usr/bin/which pacman 2>/dev/null`
+  if [ "X$PacB" != "X" ];then
+  {
+  echo $PacB > $DIR/tmp/Pac-bin-$ID.txt
+  domainP=`/bin/awk -F/ '{print NF}' $DIR/tmp/Pac-bin-$ID.txt`
+  domain=`expr $domainP - 2`
+  Pacman=`/bin/cut -d/ -f1-$domain $DIR/tmp/Pac-bin-$ID.txt`
+  }
+  else
+   {
+   Pacman=$APP/$USER/pacman-3.21
+   ./auto-pacman-inst $SITE $APP &
+    }
+  fi
+
+  #get $Signature for new sites
+  globus-job-run $SITE /bin/mkdir $APP/$USER 2>/dev/null
+  globus-url-copy file://$DIR/auto-getsignature gsiftp://$SITE$APP/$USER/auto-getsignature 2>/dev/null && globus-job-run $SITE  /bin/chmod +x  $APP/$USER/auto-getsignature 
+  Signature=`globus-job-run $SITE  $APP/$USER/auto-getsignature 2>/dev/null` && echo "$SITE $APP $Pacman $Signature $DATA  $LRM  $NAME" >> $DIR/tmp/$GRID-$VO-new-sites.txt
+
+  }
+  fi
+ done
+
+#add the new sites to the .osg-sites.cache
+/bin/cat $DIR/tmp/$GRID-$VO-new-sites.txt  >> $DIR/bin/.osg-sites.cache 2>/dev/null
+
+######update-sites-xml
+  #globus-job-run $SITE /bin/mkdir -p $DATA/$VO/$USER 2>/dev/null &
+NewFile=$DIR/tmp/$GRID-$VO-new-sites.txt
+i=1
+for NSITE in $(awk '{print $1}' $NewFile)
+  do
+{
+  SITE=$NSITE
+  NAME=`sed -n "$i"p $NewFile | awk '{print $7}'`
+  LRM=`sed -n "$i"p $NewFile |awk '{print $6}'`
+  DATA=`sed -n "$i"p $NewFile |awk '{print $5}'` 
+  SitesXML="$DIR/bin/.swift-sites.xml"
+  Line1="<pool handle=\"$NAME\" sysinfo=\"INTEL32::LINUX\">"
+  Line2="<gridftp  url=\"gsiftp://$SITE\" storage=\"$DATA/\$VO/\$USER\" major=\"2\" minor=\"2\" patch=\"4\">"
+  Line3="</gridftp>"
+  Line4="<jobmanager universe=\"vanilla\" url=\"$SITE/jobmanager-$LRM\" major=\"2\" minor=\"2\" patch=\"4\" />"
+  Line5="<workdirectory >$DATA/\$VO/\$USER</workdirectory>"
+  Line6="</pool>"
+  sed -i "s,</config>,$Line1,g" $SitesXML
+  echo $Line2 >> $SitesXML
+  echo $Line3 >> $SitesXML
+  echo $Line4 >> $SitesXML
+  echo $Line5 >> $SitesXML
+  echo $Line6 >> $SitesXML
+  echo ""     >> $SitesXML
+  echo "</config>" >> $SitesXML
+
+  i=`expr $i + 1`
+
+}
+  done


Property changes on: SwiftApps/adem-osg/bin/auto-update-sites
___________________________________________________________________
Name: svn:executable
   + 



From noreply at svn.ci.uchicago.edu  Mon Oct 20 21:17:06 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 20 Oct 2008 21:17:06 -0500 (CDT)
Subject: [Swift-commit] r2308 - SwiftApps/adem-osg/bin
Message-ID: <20081021021706.1EA4422814F@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-20 21:17:05 -0500 (Mon, 20 Oct 2008)
New Revision: 2308

Added:
   SwiftApps/adem-osg/bin/filter
Log:
New script import

Added: SwiftApps/adem-osg/bin/filter
===================================================================
--- SwiftApps/adem-osg/bin/filter	                        (rev 0)
+++ SwiftApps/adem-osg/bin/filter	2008-10-21 02:17:05 UTC (rev 2308)
@@ -0,0 +1,17 @@
+#/bin/bash
+#To filter the sites.xml or tc.data file
+loop=0
+
+#if [ X$Pattern = X ]; then
+#   echo "Please input the grid site name as the parameter"
+#   exit
+#fi
+while [ $# -ne 0 ]
+do
+  Pattern=$1
+  LineS=`cat $ADEM_HOME/bin/.swift-sites.xml | sed -n "/${Pattern}/=" 2>/dev/null`
+  LineE=`expr $LineS + 6`
+  cat $ADEM_HOME/bin/.swift-sites.xml | sed  -n "${LineS},${LineE}p" 2>/dev/null
+  shift
+done
+


Property changes on: SwiftApps/adem-osg/bin/filter
___________________________________________________________________
Name: svn:executable
   + 



From noreply at svn.ci.uchicago.edu  Mon Oct 20 21:19:07 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 20 Oct 2008 21:19:07 -0500 (CDT)
Subject: [Swift-commit] r2309 - SwiftApps/adem-osg/bin
Message-ID: <20081021021907.1F0A522814F@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-20 21:19:06 -0500 (Mon, 20 Oct 2008)
New Revision: 2309

Removed:
   SwiftApps/adem-osg/bin/auto-get-sites
Log:
sites

Deleted: SwiftApps/adem-osg/bin/auto-get-sites
===================================================================
--- SwiftApps/adem-osg/bin/auto-get-sites	2008-10-21 02:17:05 UTC (rev 2308)
+++ SwiftApps/adem-osg/bin/auto-get-sites	2008-10-21 02:19:06 UTC (rev 2309)
@@ -1,66 +0,0 @@
-#/bin/bash
-# To get the available grid sites
-if [ "X${ADEM_HOME}" = "X" ]; then
-    echo "You must set your ADEM_HOME variable! Then continue" 1>&2
-fi
-
-CDATE=`date +%F-%R`
-DIR="$ADEM_HOME"
-USER=`whoami`
-# get the grid computing sites
-GRID=$1
-VO=$2
-if [ "X$GRID" == "X" ]; then
-  echo "(1)Please input the GRID name as the first parameter, which can be:"
-  echo "osg osg-itb "
-  echo "(2)If necessary, please input the virtual organization name as the second parameter.Just use a space to separate the two parameters. For osg, vo can be:"
-  echo "CDF CMS CompBioGrid DES DOSAR DZero Engage Fermilab fMRI GADU geant4 GLOW GPN GRASE GridChem GridEx GROW i2u2 iVDGL LIGO mariachi MIS  nanoHUB NWICG  Ops  OSG OSGEDU SDSS STAR USATLAS"
-  exit 
-fi
-
-/usr/bin/test -d $DIR/tmp
-  if [ $? -ne 0 ];then
-  /bin/mkdir $DIR/tmp
-  #else
-  #/bin/rm -i -r $DIR/tmp/*
-  fi
-
-  if [ "X$VO" == "X" ]; then
-  {
-  /usr/bin/wget -O $DIR/tmp/$GRID-sitelist.txt "http://vors.grid.iu.edu/cgi-bin/tindex.cgi?grid=$GRID" 1>/dev/null 2>/dev/null 
-   }
-  else
-   {
-   /usr/bin/wget -O $DIR/tmp/$GRID-sitelist.txt "http://vors.grid.iu.edu/cgi-bin/tindex.cgi?grid=$GRID&VO=$VO" 1>/dev/null 2>/dev/null
-   }
- fi
- 
-/bin/cat $DIR/tmp/$GRID-sitelist.txt |grep compute |grep PASS |awk -F, {'print $3'} > $DIR/tmp/$GRID-$VO-compute-sites.txt
-
-n=0
-for i in $(/bin/awk -F: '{print $1}' $DIR/tmp/$GRID-$VO-compute-sites.txt)
-do
-  globusrun -a -r $i/jobmanager-fork 1>/dev/null && echo $i >>  $DIR/tmp/$GRID-$VO-sites-authentication-success-$CDATE.txt & 
-  n=`expr $n + 1`
-done
-n=`expr $n / 10`
-n=`expr $n + 10`
-sleep $n
- 
-m=0 
-if [ -f  $DIR/tmp/$GRID-$VO-sites-authentication-success-$CDATE.txt ]; then
- for SITE in $(cat $DIR/tmp/$GRID-$VO-sites-authentication-success-$CDATE.txt)
- do
-  /bin/grep $SITE $DIR/bin/.osg-sites.cache >> $DIR/tmp/$GRID-$VO-avail-sites-$CDATE.txt &&  m=`expr $m + 1`  
- done
-  echo "Now it's `date +%D-%R`.Altogether,there are $m successful available grid sites."
-  echo "The available sites file for the application deployment is: $DIR/tmp/$GRID-$VO-avail-sites-$CDATE.txt"
-fi
-
-#generate the swift-sites.xml
-cp $DIR/bin/.swift-sites.xml $DIR/swift-execution-example/swift-sites.xml
-sed -i "s,\$VO/\$USER,$VO/$USER,g" $DIR/swift-execution-example/swift-sites.xml
-echo "The general sites file for swift execution is: $DIR/swift-execution-example/swift-sites.xml"
-
-#invoke the sites cache and .swift-sites.xml update process
-#$DIR/bin/auto-update-sites $GRID $VO $DIR/tmp/$GRID-$VO-sites-authentication-success-$CDATE.txt &



From noreply at svn.ci.uchicago.edu  Mon Oct 20 21:19:25 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 20 Oct 2008 21:19:25 -0500 (CDT)
Subject: [Swift-commit] r2310 - SwiftApps/adem-osg/bin
Message-ID: <20081021021925.1E11422814F@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-20 21:19:24 -0500 (Mon, 20 Oct 2008)
New Revision: 2310

Added:
   SwiftApps/adem-osg/bin/auto-get-sites
Log:
New script import

Added: SwiftApps/adem-osg/bin/auto-get-sites
===================================================================
--- SwiftApps/adem-osg/bin/auto-get-sites	                        (rev 0)
+++ SwiftApps/adem-osg/bin/auto-get-sites	2008-10-21 02:19:24 UTC (rev 2310)
@@ -0,0 +1,66 @@
+#/bin/bash
+# To get the available grid sites
+if [ "X${ADEM_HOME}" = "X" ]; then
+    echo "You must set your ADEM_HOME variable! Then continue" 1>&2
+fi
+
+CDATE=`date +%F-%R`
+DIR="$ADEM_HOME"
+USER=`whoami`
+# get the grid computing sites
+GRID=$1
+VO=$2
+if [ "X$GRID" == "X" ]; then
+  echo "(1)Please input the GRID name as the first parameter, which can be:"
+  echo "osg osg-itb "
+  echo "(2)If necessary, please input the virtual organization name as the second parameter.Just use a space to separate the two parameters. For osg, vo can be:"
+  echo "CDF CMS CompBioGrid DES DOSAR DZero Engage Fermilab fMRI GADU geant4 GLOW GPN GRASE GridChem GridEx GROW i2u2 iVDGL LIGO mariachi MIS  nanoHUB NWICG  Ops  OSG OSGEDU SDSS STAR USATLAS"
+  exit 
+fi
+
+/usr/bin/test -d $DIR/tmp
+  if [ $? -ne 0 ];then
+  /bin/mkdir $DIR/tmp
+  #else
+  #/bin/rm -i -r $DIR/tmp/*
+  fi
+
+  if [ "X$VO" == "X" ]; then
+  {
+  /usr/bin/wget -O $DIR/tmp/$GRID-sitelist.txt "http://vors.grid.iu.edu/cgi-bin/tindex.cgi?grid=$GRID" 1>/dev/null 2>/dev/null 
+   }
+  else
+   {
+   /usr/bin/wget -O $DIR/tmp/$GRID-sitelist.txt "http://vors.grid.iu.edu/cgi-bin/tindex.cgi?grid=$GRID&VO=$VO" 1>/dev/null 2>/dev/null
+   }
+ fi
+ 
+/bin/cat $DIR/tmp/$GRID-sitelist.txt |grep compute |grep PASS |awk -F, {'print $3'} > $DIR/tmp/$GRID-$VO-compute-sites.txt
+
+n=0
+for i in $(/bin/awk -F: '{print $1}' $DIR/tmp/$GRID-$VO-compute-sites.txt)
+do
+  globusrun -a -r $i/jobmanager-fork 1>/dev/null && echo $i >>  $DIR/tmp/$GRID-$VO-sites-authentication-success-$CDATE.txt & 
+  n=`expr $n + 1`
+done
+n=`expr $n / 10`
+n=`expr $n + 10`
+sleep $n
+ 
+m=0 
+if [ -f  $DIR/tmp/$GRID-$VO-sites-authentication-success-$CDATE.txt ]; then
+ for SITE in $(cat $DIR/tmp/$GRID-$VO-sites-authentication-success-$CDATE.txt)
+ do
+  /bin/grep $SITE $DIR/bin/.osg-sites.cache >> $DIR/tmp/$GRID-$VO-avail-sites-$CDATE.txt &&  m=`expr $m + 1`  
+ done
+  echo "Now it's `date +%D-%R`.Altogether,there are $m successful available grid sites."
+  echo "The available sites file for the application deployment is: $DIR/tmp/$GRID-$VO-avail-sites-$CDATE.txt"
+fi
+
+#generate the swift-sites.xml
+cp $DIR/bin/.swift-sites.xml $DIR/swift-execution-example/swift-sites.xml
+sed -i "s,\$VO/\$USER,$VO/$USER,g" $DIR/swift-execution-example/swift-sites.xml
+echo "The general sites file for swift execution is: $DIR/swift-execution-example/swift-sites.xml"
+
+#invoke the sites cache and .swift-sites.xml update process
+#$DIR/bin/auto-update-sites $GRID $VO $DIR/tmp/$GRID-$VO-sites-authentication-success-$CDATE.txt &


Property changes on: SwiftApps/adem-osg/bin/auto-get-sites
___________________________________________________________________
Name: svn:executable
   + 



From noreply at svn.ci.uchicago.edu  Mon Oct 20 21:21:58 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 20 Oct 2008 21:21:58 -0500 (CDT)
Subject: [Swift-commit] r2311 -
	SwiftApps/adem-osg/swift-execution-example/dock
Message-ID: <20081021022158.D8ECA22814F@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-20 21:21:58 -0500 (Mon, 20 Oct 2008)
New Revision: 2311

Removed:
   SwiftApps/adem-osg/swift-execution-example/dock/prepare-for-dock-swift-submit
Log:
prepare

Deleted: SwiftApps/adem-osg/swift-execution-example/dock/prepare-for-dock-swift-submit
===================================================================
--- SwiftApps/adem-osg/swift-execution-example/dock/prepare-for-dock-swift-submit	2008-10-21 02:19:24 UTC (rev 2310)
+++ SwiftApps/adem-osg/swift-execution-example/dock/prepare-for-dock-swift-submit	2008-10-21 02:21:58 UTC (rev 2311)
@@ -1,48 +0,0 @@
-#/bin/sh
-DIR="$ADEM_HOME"
-USER=`whoami`
-i=1
-
-VO=$1
-SITE_FILE=$2
-
-if [ "X$VO" == "X" ]; then
-  {
-  echo "Please input the virtual organization name as the first parameter, and the grid sites file as the second parameter.Just use a space to separate the two parameters. For osg, vo can be:"
-  echo "CDF CMS CompBioGrid DES DOSAR DZero Engage Fermilab fMRI GADU geant4 GLOW GPN GRASE GridChem GridEx GROW i2u2 iVDGL LIGO mariachi MIS  nanoHUB NWICG  Ops  OSG OSGEDU SDSS STAR USATLAS"
-  }
-  exit
-fi
-
-if [ "$SITE_FILE" == "" ]; then
-  {
-  echo "Oh,no grid sites file for the application deployment.Please input the grid sites file as the parameter.You can use auto-get-sites to create the available grid sites automatically."
-  }
-  exit
-fi
-
-echo "#sitename transformation  path    INSTALLED       platform        profiles" > $DIR/swift-execution-example/dock/dock-tc.data
-for SITE in $(awk '{print $1}' $SITE_FILE)
-do
-  echo $i
-  echo $SITE
-  APP=`sed -n "$i"p $SITE_FILE | awk {'print $2'}`
-  #PACMAN=`sed -n "$i"p $SITE_FILE | awk {'print $3'}`
-  DATA=`sed -n "$i"p $SITE_FILE | awk {'print $5'}`
-  NAME=`sed -n "$i"p $SITE_FILE | awk {'print $7'}`
-
-#(1) make an individual work directory in the VO
-#(2) To transfer the input files for DOCK application
-  globus-job-run $SITE /bin/mkdir -p $DATA/$VO/$USER 2>/dev/null && globus-url-copy  file:///home/houzx/dock-run/dockKAGxNL.tar.gz  gsiftp://$SITE$DATA/$VO/$USER/dockKAGxNL.tar.gz &
-
-#(3) To modify and transfer the executable wrapper script for DOCK application
-  /bin/cp wrapper-bin/rundock-template wrapper-bin/rundock-$SITE
-  sed -i "s,\$DATA/\$VO/\$USER,$DATA/$VO/$USER,g"  wrapper-bin/rundock-$SITE
-  sed -i "s,\$APP/osg,$APP/osg,g"  wrapper-bin/rundock-$SITE
-  globus-url-copy  file:///home/houzx/dock-run/bin/rundock-$SITE gsiftp://$SITE$DATA/$VO/$USER/rundock && globus-job-run  $SITE /bin/chmod +x $DATA/$VO/$USER/rundock &
-  echo"$NAME	rundock		$DATA/$VO/$USER/rundock	INSTALLED	INTEL32::LINUX	null" >> $DIR/swift-execution-example/dock/dock-tc.data
- 
-  i=`expr $i + 1`
-done
-
-echo "The tc.data file for the swift execution of DOCK application is: $DIR/swift-execution-example/dock/dock-tc.data"



From noreply at svn.ci.uchicago.edu  Mon Oct 20 21:22:21 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 20 Oct 2008 21:22:21 -0500 (CDT)
Subject: [Swift-commit] r2312 -
	SwiftApps/adem-osg/swift-execution-example/dock
Message-ID: <20081021022221.BC0FF22814F@www.ci.uchicago.edu>

Author: houzx
Date: 2008-10-20 21:22:21 -0500 (Mon, 20 Oct 2008)
New Revision: 2312

Added:
   SwiftApps/adem-osg/swift-execution-example/dock/prepare-for-dock-swift-submit
Log:
prepare

Added: SwiftApps/adem-osg/swift-execution-example/dock/prepare-for-dock-swift-submit
===================================================================
--- SwiftApps/adem-osg/swift-execution-example/dock/prepare-for-dock-swift-submit	                        (rev 0)
+++ SwiftApps/adem-osg/swift-execution-example/dock/prepare-for-dock-swift-submit	2008-10-21 02:22:21 UTC (rev 2312)
@@ -0,0 +1,48 @@
+#/bin/sh
+DIR="$ADEM_HOME"
+USER=`whoami`
+i=1
+
+VO=$1
+SITE_FILE=$2
+
+if [ "X$VO" == "X" ]; then
+  {
+  echo "Please input the virtual organization name as the first parameter, and the grid sites file as the second parameter.Just use a space to separate the two parameters. For osg, vo can be:"
+  echo "CDF CMS CompBioGrid DES DOSAR DZero Engage Fermilab fMRI GADU geant4 GLOW GPN GRASE GridChem GridEx GROW i2u2 iVDGL LIGO mariachi MIS  nanoHUB NWICG  Ops  OSG OSGEDU SDSS STAR USATLAS"
+  }
+  exit
+fi
+
+if [ "$SITE_FILE" == "" ]; then
+  {
+  echo "Oh,no grid sites file for the application deployment.Please input the grid sites file as the parameter.You can use auto-get-sites to create the available grid sites automatically."
+  }
+  exit
+fi
+
+echo "#sitename transformation  path    INSTALLED       platform        profiles" > $DIR/swift-execution-example/dock/dock-tc.data
+for SITE in $(awk '{print $1}' $SITE_FILE)
+do
+  echo $i
+  echo $SITE
+  APP=`sed -n "$i"p $SITE_FILE | awk {'print $2'}`
+  #PACMAN=`sed -n "$i"p $SITE_FILE | awk {'print $3'}`
+  DATA=`sed -n "$i"p $SITE_FILE | awk {'print $5'}`
+  NAME=`sed -n "$i"p $SITE_FILE | awk {'print $7'}`
+
+#(1) make an individual work directory in the VO
+#(2) To transfer the input files for DOCK application
+  globus-job-run $SITE /bin/mkdir -p $DATA/$VO/$USER 2>/dev/null && globus-url-copy  file:///disks/tp-gpfs/scratch/houzx/dock-run/dockKAGxNL.tar.gz  gsiftp://$SITE$DATA/$VO/$USER/dockKAGxNL.tar.gz &
+
+#(3) To modify and transfer the executable wrapper script for DOCK application
+  /bin/cp $DIR/swift-execution-example/dock/wrapper-bin/rundock-template $DIR/swift-execution-example/dock/wrapper-bin/rundock-$SITE
+  sed -i "s,\$DATA/\$VO/\$USER,$DATA/$VO/$USER,g"  $DIR/swift-execution-example/dock/wrapper-bin/rundock-$SITE
+  sed -i "s,\$APP/osg,$APP/osg,g"  $DIR/swift-execution-example/dock/wrapper-bin/rundock-$SITE
+  sed -i "s,\$SITES,$SITE,g"  $DIR/swift-execution-example/dock/wrapper-bin/rundock-$SITE
+  globus-url-copy  file://$DIR/swift-execution-example/dock/wrapper-bin/rundock-$SITE gsiftp://$SITE$DATA/$VO/$USER/rundock && globus-job-run  $SITE /bin/chmod +x $DATA/$VO/$USER/rundock && echo "$NAME	rundock		$DATA/$VO/$USER/rundock	INSTALLED	INTEL32::LINUX	null" >> $DIR/swift-execution-example/dock/dock-tc.data
+ 
+  i=`expr $i + 1`
+done
+
+echo "The tc.data file for the swift execution of DOCK application is: $DIR/swift-execution-example/dock/dock-tc.data"


Property changes on: SwiftApps/adem-osg/swift-execution-example/dock/prepare-for-dock-swift-submit
___________________________________________________________________
Name: svn:executable
   + 



From noreply at svn.ci.uchicago.edu  Wed Oct 22 01:57:51 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 22 Oct 2008 01:57:51 -0500 (CDT)
Subject: [Swift-commit] r2313 - trunk/resources
Message-ID: <20081022065751.D839C22814F@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-22 01:57:50 -0500 (Wed, 22 Oct 2008)
New Revision: 2313

Modified:
   trunk/resources/swiftscript.g
Log:
whitespace

Modified: trunk/resources/swiftscript.g
===================================================================
--- trunk/resources/swiftscript.g	2008-10-21 02:22:21 UTC (rev 2312)
+++ trunk/resources/swiftscript.g	2008-10-22 06:57:50 UTC (rev 2313)
@@ -35,7 +35,7 @@
 StringTemplate template(String name) {
 	StringTemplate t = m_templates.getInstanceOf(name);
 	t.setAttribute("sourcelocation","line "+swiftLexer.getLine());
-    return t;
+	return t;
 }
 
 StringTemplate text(String t) {



From noreply at svn.ci.uchicago.edu  Wed Oct 22 02:03:24 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 22 Oct 2008 02:03:24 -0500 (CDT)
Subject: [Swift-commit] r2314 - trunk/resources
Message-ID: <20081022070324.49A95228196@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-22 02:03:23 -0500 (Wed, 22 Oct 2008)
New Revision: 2314

Modified:
   trunk/resources/swiftscript.g
Log:
whitespace fix the right way round

Modified: trunk/resources/swiftscript.g
===================================================================
--- trunk/resources/swiftscript.g	2008-10-22 06:57:50 UTC (rev 2313)
+++ trunk/resources/swiftscript.g	2008-10-22 07:03:23 UTC (rev 2314)
@@ -29,13 +29,13 @@
 /** TODO this can perhaps be extracted from the superclass, but I don't
     have javadocs available at the time of writing. */
 public void setSwiftLexer(SwiftScriptLexer sl) {
-	swiftLexer = sl;
+    swiftLexer = sl;
 }
 
 StringTemplate template(String name) {
-	StringTemplate t = m_templates.getInstanceOf(name);
-	t.setAttribute("sourcelocation","line "+swiftLexer.getLine());
-	return t;
+    StringTemplate t = m_templates.getInstanceOf(name);
+    t.setAttribute("sourcelocation","line "+swiftLexer.getLine());
+    return t;
 }
 
 StringTemplate text(String t) {



From noreply at svn.ci.uchicago.edu  Wed Oct 22 12:28:05 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 22 Oct 2008 12:28:05 -0500 (CDT)
Subject: [Swift-commit] r2315 - in trunk: resources tests/language-behaviour
Message-ID: <20081022172805.525BC228181@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-22 12:28:04 -0500 (Wed, 22 Oct 2008)
New Revision: 2315

Added:
   trunk/tests/language-behaviour/0011-echo.out.expected
   trunk/tests/language-behaviour/0011-echo.swift
Modified:
   trunk/resources/swiftscript.g
Log:
alternative app block syntax

Modified: trunk/resources/swiftscript.g
===================================================================
--- trunk/resources/swiftscript.g	2008-10-22 07:03:23 UTC (rev 2314)
+++ trunk/resources/swiftscript.g	2008-10-22 17:28:04 UTC (rev 2315)
@@ -174,6 +174,7 @@
 
 // this is a declaration, but not sorted out the predications yet to
 // group it into a decl block
+    | ("app") => d=appproceduredecl {code.setAttribute("functions",d);}
     | (predictProceduredecl) => d=proceduredecl {code.setAttribute("functions", d);}
     ;
 
@@ -371,6 +372,49 @@
         }
     ;
 
+appproceduredecl returns [StringTemplate code=template("function")]
+{StringTemplate f=null;
+ StringTemplate app=template("app");
+ StringTemplate exec=null; }
+    :   "app"
+        ( LPAREN
+        f=formalParameter
+        {
+        f.setAttribute("outlink", "true");
+        code.setAttribute("outputs", f);
+        }
+        (   COMMA f=formalParameter
+            {
+            f.setAttribute("outlink", "true");
+            code.setAttribute("outputs", f);
+            }
+        )*
+        RPAREN )?
+        id:ID {currentFunctionName=id.getText();} LPAREN
+        (   f=formalParameter
+            {
+            code.setAttribute("inputs", f);
+            }
+            (   COMMA f=formalParameter
+                {
+                code.setAttribute("inputs", f);
+                }
+            )*
+        )?
+        RPAREN
+        LCURLY
+        exec=declarator
+        {app.setAttribute("exec",exec);}
+        ( appArg[app] )* SEMI
+        {code.setAttribute("config",app);}
+        RCURLY
+        {
+        code.setAttribute("name", id.getText());
+        currentFunctionName=null;
+        }
+    ;
+
+
 // TODO in here, why do we have an | between LBRACKBRACK and ASSIGN?
 // does this mean that we don't have array initialisation in formal
 // params? this wouldn't surprise me given the previous treatment

Added: trunk/tests/language-behaviour/0011-echo.out.expected
===================================================================
--- trunk/tests/language-behaviour/0011-echo.out.expected	                        (rev 0)
+++ trunk/tests/language-behaviour/0011-echo.out.expected	2008-10-22 17:28:04 UTC (rev 2315)
@@ -0,0 +1 @@
+hello

Added: trunk/tests/language-behaviour/0011-echo.swift
===================================================================
--- trunk/tests/language-behaviour/0011-echo.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/0011-echo.swift	2008-10-22 17:28:04 UTC (rev 2315)
@@ -0,0 +1,10 @@
+type messagefile;
+
+app (messagefile t) greeting() { 
+        echo "hello" stdout=@filename(t);
+}
+
+messagefile outfile <"0011-echo.out">;
+
+outfile = greeting();
+



From noreply at svn.ci.uchicago.edu  Thu Oct 23 14:55:14 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 23 Oct 2008 14:55:14 -0500 (CDT)
Subject: [Swift-commit] r2316 - SwiftApps/SIDGrid/config
Message-ID: <20081023195514.D93322281A1@www.ci.uchicago.edu>

Author: skenny
Date: 2008-10-23 14:55:14 -0500 (Thu, 23 Oct 2008)
New Revision: 2316

Added:
   SwiftApps/SIDGrid/config/coaster_ranger.xml
Log:
for running with coasters

Added: SwiftApps/SIDGrid/config/coaster_ranger.xml
===================================================================
--- SwiftApps/SIDGrid/config/coaster_ranger.xml	                        (rev 0)
+++ SwiftApps/SIDGrid/config/coaster_ranger.xml	2008-10-23 19:55:14 UTC (rev 2316)
@@ -0,0 +1,17 @@
+<?xml version="1.0" encoding="ISO-8859-1"?>
+<config xmlns="http://www.griphyn.org/chimera/GVDS-PoolConfig" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.griphyn.org/chimera/GVDS-PoolConfig http://www.griphyn.org/chimera/sc-1.4.xsd" version="1.4">
+ <import file="sys.xml"/>
+<import file="vdl-lib.xml"/>
+<set name="username">
+  <arg><vdl:new type="string" value="user"/></arg>
+</set>
+  <!-- RANGER @ tg-login.ranger.tacc.teragrid.org -->
+  <pool handle="RANGER">
+    <profile namespace="karajan" key="initialScore">1</profile>
+    <profile namespace="karajan" key="jobThrottle">8</profile>
+    <filesystem provider="coaster" url="gt2://gatekeeper.ranger.tacc.teragrid.org"/>
+    <profile namespace="globus" key="coastersPerNode">16</profile>
+    <execution provider="coaster" url="gatekeeper.ranger.tacc.teragrid.org" jobManager="gt2:gt2:SGE"/>
+    <workdirectory>/work/00926/tg459516/sidgrid_out/{username}</workdirectory>
+  </pool>
+</config>



From noreply at svn.ci.uchicago.edu  Fri Oct 24 02:16:01 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 24 Oct 2008 02:16:01 -0500 (CDT)
Subject: [Swift-commit] r2317 - trunk/src/org/griphyn/vdl/karajan/lib
Message-ID: <20081024071601.1686E228199@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-24 02:15:59 -0500 (Fri, 24 Oct 2008)
New Revision: 2317

Modified:
   trunk/src/org/griphyn/vdl/karajan/lib/SetFieldValue.java
Log:
runtime warning when using array assignment outside of a variable initialization; this does not work.

Modified: trunk/src/org/griphyn/vdl/karajan/lib/SetFieldValue.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/SetFieldValue.java	2008-10-23 19:55:14 UTC (rev 2316)
+++ trunk/src/org/griphyn/vdl/karajan/lib/SetFieldValue.java	2008-10-24 07:15:59 UTC (rev 2317)
@@ -4,6 +4,7 @@
 package org.griphyn.vdl.karajan.lib;
 
 import org.apache.log4j.Logger;
+import org.griphyn.vdl.karajan.PairIterator;
 import org.globus.cog.karajan.arguments.Arg;
 import org.globus.cog.karajan.stack.VariableStack;
 import org.globus.cog.karajan.workflow.ExecutionException;
@@ -35,6 +36,10 @@
 // for type conversion here; but would be useful to have
 // type checking.
 				// leaf.setValue(internalValue(leaf.getType(), value));
+				if( (value instanceof DSHandle && ((DSHandle)value).getType().isArray()) || (value instanceof PairIterator)) {
+					logger.warn("Warning: array assignment outside of initialisation does not work correctly.");
+				}
+
 				leaf.setValue(value);
 				closeShallow(stack, leaf);
 			}



From noreply at svn.ci.uchicago.edu  Fri Oct 24 03:12:59 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 24 Oct 2008 03:12:59 -0500 (CDT)
Subject: [Swift-commit] r2318 - trunk/src/org/griphyn/vdl/karajan/lib
Message-ID: <20081024081259.2D8C8228199@www.ci.uchicago.edu>

Author: benc
Date: 2008-10-24 03:12:58 -0500 (Fri, 24 Oct 2008)
New Revision: 2318

Modified:
   trunk/src/org/griphyn/vdl/karajan/lib/CreateArray.java
Log:
tidy up error reporting in CreateArray

Modified: trunk/src/org/griphyn/vdl/karajan/lib/CreateArray.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/CreateArray.java	2008-10-24 07:15:59 UTC (rev 2317)
+++ trunk/src/org/griphyn/vdl/karajan/lib/CreateArray.java	2008-10-24 08:12:58 UTC (rev 2318)
@@ -82,9 +82,6 @@
 			return handle;
 		}
 		catch (Exception e) {
-System.err.println("OOK");
-e.printStackTrace();
-System.err.println("ENDOOK");
 			throw new ExecutionException(e);
 		}
 	}