From noreply at svn.ci.uchicago.edu  Thu Jul  3 04:40:39 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu,  3 Jul 2008 04:40:39 -0500 (CDT)
Subject: [Swift-commit] r2082 - log-processing
Message-ID: <20080703094039.2E8049B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-03 04:40:38 -0500 (Thu, 03 Jul 2008)
New Revision: 2082

Modified:
   log-processing/table-jobs-sites
Log:
handle JOB_CANCELLED state

Modified: log-processing/table-jobs-sites
===================================================================
--- log-processing/table-jobs-sites	2008-06-30 19:04:31 UTC (rev 2081)
+++ log-processing/table-jobs-sites	2008-07-03 09:40:38 UTC (rev 2082)
@@ -4,7 +4,7 @@
 # file in a 2-d html table, from execute2.event
 # 1193173122 1 goodmonkey-3xu5anhi FAILED localhost
 
-STATES="JOB_START JOB_END APPLICATION_EXCEPTION"
+STATES="JOB_START JOB_END APPLICATION_EXCEPTION JOB_CANCELED"
 
 echo "<table border=\"1\">"
 



From noreply at svn.ci.uchicago.edu  Fri Jul  4 12:34:16 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri,  4 Jul 2008 12:34:16 -0500 (CDT)
Subject: [Swift-commit] r2083 - www/inc
Message-ID: <20080704173416.84F0A9B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-04 12:34:15 -0500 (Fri, 04 Jul 2008)
New Revision: 2083

Modified:
   www/inc/downloads_sidebar.php
Log:
fix broken quickstart guide link discovered by Timothy Freeman

Modified: www/inc/downloads_sidebar.php
===================================================================
--- www/inc/downloads_sidebar.php	2008-07-03 09:40:38 UTC (rev 2082)
+++ www/inc/downloads_sidebar.php	2008-07-04 17:34:15 UTC (rev 2083)
@@ -1,6 +1,6 @@
 <h3>GETTING STARTED </h3>
 <p>
-<strong>Swift Quickstart Guide</strong> [<a href="#">html</a>]</p>
+<strong>Swift Quickstart Guide</strong> [<a href="http://www.ci.uchicago.edu/swift/guides/quickstartguide.php">html</a>]</p>
 <p>Use the Quickstart Guide to help you install and configure swift and run a simple 'Hello World' example.</p>
 
 <h4>BUG REPORTS  </h4>



From noreply at svn.ci.uchicago.edu  Sun Jul  6 18:06:24 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sun,  6 Jul 2008 18:06:24 -0500 (CDT)
Subject: [Swift-commit] r2084 - SwiftApps/SIDGrid/config
Message-ID: <20080706230624.3ACCF9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-06 18:06:23 -0500 (Sun, 06 Jul 2008)
New Revision: 2084

Added:
   SwiftApps/SIDGrid/config/sites_ranger.xml
Log:
adding ranger

Added: SwiftApps/SIDGrid/config/sites_ranger.xml
===================================================================
--- SwiftApps/SIDGrid/config/sites_ranger.xml	                        (rev 0)
+++ SwiftApps/SIDGrid/config/sites_ranger.xml	2008-07-06 23:06:23 UTC (rev 2084)
@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="ISO-8859-1"?>
+<config xmlns="http://www.griphyn.org/chimera/GVDS-PoolConfig" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.griphyn.org/chimera/GVDS-PoolConfig http://www.griphyn.org/chimera/sc-1.4.xsd" version="1.4">
+<import file="sys.xml"/>
+<import file="vdl-lib.xml"/>
+<set name="username">
+  <arg><vdl:new type="string" value="user"/></arg>
+</set>
+  <!-- RANGER @ tg-login.ranger.tacc.teragrid.org -->
+  <pool handle="RANGER" sysinfo="INTEL64::LINUX">
+    <gridftp url="gsiftp://gridftp.ranger.tacc.teragrid.org" storage="/work/00043/tg457040/sidgrid_out/{username}" major="2" minor="4" patch="3"/>
+    <jobmanager universe="vanilla" url="tg-login.ranger.tacc.teragrid.org/jobmanager-sge" major="2" minor="4" patch="3"/>
+    <workdirectory>sidgrid_out/{username}</workdirectory>
+  </pool>
+</config>



From noreply at svn.ci.uchicago.edu  Mon Jul  7 04:11:55 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 04:11:55 -0500 (CDT)
Subject: [Swift-commit] r2085 - trunk/docs
Message-ID: <20080707091155.EE31A9B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-07 04:11:54 -0500 (Mon, 07 Jul 2008)
New Revision: 2085

Modified:
   trunk/docs/userguide.xml
Log:
note on using mpi

Modified: trunk/docs/userguide.xml
===================================================================
--- trunk/docs/userguide.xml	2008-07-06 23:06:23 UTC (rev 2084)
+++ trunk/docs/userguide.xml	2008-07-07 09:11:54 UTC (rev 2085)
@@ -2360,6 +2360,49 @@
 </para>
 
 		</section>
+		<section><title>Launching MPI jobs from Swift</title>
+<para>
+Here is an example of running a simple MPI program.
+</para>
+<para>
+In SwiftScript, we make an invocation that does not look any different
+from any other invocation. In the below code, we do not have any input
+files, and have two output files on stdout and stderr:
+<programlisting>
+type file;
+
+(file o, file e) p() { 
+    app {
+        mpi stdout=@filename(o) stderr=@filename(e);
+    }
+}
+
+file mpiout &lt;"mpi.out">;
+file mpierr &lt;"mpi.err">;
+
+(mpiout, mpierr) = p();
+</programlisting>
+</para>
+<para>
+Now we define how 'mpi' will run in tc.data:
+<programlisting>
+tguc    mpi             /home/benc/mpi/mpi.sh   INSTALLED       INTEL32::LINUX GLOBUS::host_xcount=3
+</programlisting>
+</para>
+<para>
+mpi.sh is a wrapper script that launches the MPI program. It must be installed
+on the remote site:
+<programlisting>
+#!/bin/bash
+mpirun -np 3 -machinefile $PBS_NODEFILE /home/benc/mpi/a.out 
+</programlisting>
+</para>
+<para>
+Because of the way that Swift runs its server side code, provider-specific
+MPI modes (such as GRAM jobType=mpi) should not be used. Instead, the
+mpirun command should be explicitly invoked.
+</para>
+		</section>
 	</section>
 
 	<section><title>The Site Catalog - sites.xml</title>



From noreply at svn.ci.uchicago.edu  Mon Jul  7 07:32:43 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 07:32:43 -0500 (CDT)
Subject: [Swift-commit] r2086 - trunk/tests/misc
Message-ID: <20080707123243.14DF79B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-07 07:32:42 -0500 (Mon, 07 Jul 2008)
New Revision: 2086

Modified:
   trunk/tests/misc/run
Log:
final success message for misc. tests

Modified: trunk/tests/misc/run
===================================================================
--- trunk/tests/misc/run	2008-07-07 09:11:54 UTC (rev 2085)
+++ trunk/tests/misc/run	2008-07-07 12:32:42 UTC (rev 2086)
@@ -7,3 +7,4 @@
    exit $R
  fi
 done
+echo All misc. tests passed ok



From noreply at svn.ci.uchicago.edu  Mon Jul  7 07:34:25 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 07:34:25 -0500 (CDT)
Subject: [Swift-commit] r2087 - trunk/tests/language-behaviour
Message-ID: <20080707123425.8983C9B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-07 07:34:25 -0500 (Mon, 07 Jul 2008)
New Revision: 2087

Added:
   trunk/tests/language-behaviour/065-delay.swift
Modified:
   trunk/tests/language-behaviour/generate-tc.data
Log:
test long delay job

Added: trunk/tests/language-behaviour/065-delay.swift
===================================================================
--- trunk/tests/language-behaviour/065-delay.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/065-delay.swift	2008-07-07 12:34:25 UTC (rev 2087)
@@ -0,0 +1,9 @@
+
+delay() { 
+    app {
+        sleep "90s";
+    }
+}
+
+delay();
+

Modified: trunk/tests/language-behaviour/generate-tc.data
===================================================================
--- trunk/tests/language-behaviour/generate-tc.data	2008-07-07 12:32:42 UTC (rev 2086)
+++ trunk/tests/language-behaviour/generate-tc.data	2008-07-07 12:34:25 UTC (rev 2087)
@@ -1,5 +1,5 @@
 #!/bin/bash
 
-for cmd in wc touch; do
+for cmd in wc touch sleep; do
   echo "localhost	$cmd	$(which $cmd)	INSTALLED	INTEL32::LINUX	null" >> ./tc.data
 done



From noreply at svn.ci.uchicago.edu  Mon Jul  7 07:36:27 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 07:36:27 -0500 (CDT)
Subject: [Swift-commit] r2088 - trunk
Message-ID: <20080707123627.5D4309B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-07 07:36:26 -0500 (Mon, 07 Jul 2008)
New Revision: 2088

Modified:
   trunk/dependencies.xml
Log:
add build option for provider-wonky

Modified: trunk/dependencies.xml
===================================================================
--- trunk/dependencies.xml	2008-07-07 12:34:25 UTC (rev 2087)
+++ trunk/dependencies.xml	2008-07-07 12:36:26 UTC (rev 2088)
@@ -13,6 +13,7 @@
 		<antcall target="provider-deef"/>
 		<antcall target="provider-coaster"/>
 		<antcall target="provider-condor"/>
+		<antcall target="provider-wonky"/>
 	</target>
 	
 	<target name="provider-deef" if="with-provider-deef">
@@ -30,4 +31,9 @@
 			<property name="module" value="provider-condor"/>
 		</ant>
 	</target>
+	<target name="provider-wonky" if="with-provider-wonky">
+		<ant antfile="${main.buildfile}" target="dep">
+			<property name="module" value="provider-wonky"/>
+		</ant>
+	</target>
 </project>



From noreply at svn.ci.uchicago.edu  Mon Jul  7 07:39:22 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 07:39:22 -0500 (CDT)
Subject: [Swift-commit] r2089 - in trunk: libexec
	src/org/griphyn/vdl/karajan/lib
Message-ID: <20080707123922.80F319B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-07 07:39:21 -0500 (Mon, 07 Jul 2008)
New Revision: 2089

Added:
   trunk/src/org/griphyn/vdl/karajan/lib/IsFileBound.java
Removed:
   trunk/src/org/griphyn/vdl/karajan/lib/IsDatasetBound.java
Modified:
   trunk/libexec/vdl-lib.xml
   trunk/libexec/vdl.k
Log:
relabel isDatasetBound to isFileBound, ready to introduce datasets that are not bound to files.

Modified: trunk/libexec/vdl-lib.xml
===================================================================
--- trunk/libexec/vdl-lib.xml	2008-07-07 12:36:26 UTC (rev 2088)
+++ trunk/libexec/vdl-lib.xml	2008-07-07 12:39:21 UTC (rev 2089)
@@ -45,7 +45,7 @@
 	<export name="getFieldSubscript"><elementDef classname="org.griphyn.vdl.karajan.lib.GetFieldSubscript"/></export>
 	<export name="setFieldValue"><elementDef classname="org.griphyn.vdl.karajan.lib.SetFieldValue"/></export>
 	<export name="getFieldValue"><elementDef classname="org.griphyn.vdl.karajan.lib.GetFieldValue"/></export>
-	<export name="isDatasetBound"><elementDef classname="org.griphyn.vdl.karajan.lib.IsDatasetBound"/></export>
+	<export name="isFileBound"><elementDef classname="org.griphyn.vdl.karajan.lib.IsFileBound"/></export>
 	<export name="fileSet"><elementDef classname="org.griphyn.vdl.karajan.lib.FileSet"/></export>
 	<export name="fringePaths"><elementDef classname="org.griphyn.vdl.karajan.lib.FringePaths"/></export>
 	<export name="assign"><elementDef classname="org.griphyn.vdl.karajan.lib.Assign"/></export>

Modified: trunk/libexec/vdl.k
===================================================================
--- trunk/libexec/vdl.k	2008-07-07 12:36:26 UTC (rev 2088)
+++ trunk/libexec/vdl.k	2008-07-07 12:39:21 UTC (rev 2089)
@@ -54,7 +54,7 @@
 
 		element(stagein, [var]
 			if(
-				vdl:isDatasetBound(var) try(
+				vdl:isFileBound(var) try(
 					sequential(
 						fp := vdl:fringePaths(var)
 						try (
@@ -87,7 +87,7 @@
 
 		element(stageout, [var]
 			try(
-				if(vdl:isDatasetBound(var) 
+				if(vdl:isFileBound(var) 
 					channel:to(stageout,
 						for(path, vdl:fringePaths(var)
 							list(path, var)

Deleted: trunk/src/org/griphyn/vdl/karajan/lib/IsDatasetBound.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/IsDatasetBound.java	2008-07-07 12:36:26 UTC (rev 2088)
+++ trunk/src/org/griphyn/vdl/karajan/lib/IsDatasetBound.java	2008-07-07 12:39:21 UTC (rev 2089)
@@ -1,27 +0,0 @@
-/*
- * Created on Dec 26, 2006
- */
-package org.griphyn.vdl.karajan.lib;
-
-import org.globus.cog.karajan.arguments.Arg;
-import org.globus.cog.karajan.stack.VariableStack;
-import org.globus.cog.karajan.workflow.ExecutionException;
-import org.griphyn.vdl.mapping.AbstractDataNode;
-import org.griphyn.vdl.mapping.DSHandle;
-
-public class IsDatasetBound extends VDLFunction {
-	static {
-		setArguments(IsDatasetBound.class, new Arg[] { PA_VAR });
-	}
-
-	public Object function(VariableStack stack) throws ExecutionException {
-		DSHandle var = (DSHandle) PA_VAR.getValue(stack);
-		if (var instanceof AbstractDataNode) {
-			return Boolean.valueOf(!((AbstractDataNode) var).isPrimitive());
-		}
-		else {
-			return Boolean.FALSE;
-		}
-	}
-
-}

Added: trunk/src/org/griphyn/vdl/karajan/lib/IsFileBound.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/IsFileBound.java	                        (rev 0)
+++ trunk/src/org/griphyn/vdl/karajan/lib/IsFileBound.java	2008-07-07 12:39:21 UTC (rev 2089)
@@ -0,0 +1,27 @@
+/*
+ * Created on Dec 26, 2006
+ */
+package org.griphyn.vdl.karajan.lib;
+
+import org.globus.cog.karajan.arguments.Arg;
+import org.globus.cog.karajan.stack.VariableStack;
+import org.globus.cog.karajan.workflow.ExecutionException;
+import org.griphyn.vdl.mapping.AbstractDataNode;
+import org.griphyn.vdl.mapping.DSHandle;
+
+public class IsFileBound extends VDLFunction {
+	static {
+		setArguments(IsFileBound.class, new Arg[] { PA_VAR });
+	}
+
+	public Object function(VariableStack stack) throws ExecutionException {
+		DSHandle var = (DSHandle) PA_VAR.getValue(stack);
+		if (var instanceof AbstractDataNode) {
+			return Boolean.valueOf(!((AbstractDataNode) var).isPrimitive());
+		}
+		else {
+			return Boolean.FALSE;
+		}
+	}
+
+}



From noreply at svn.ci.uchicago.edu  Mon Jul  7 11:18:00 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 11:18:00 -0500 (CDT)
Subject: [Swift-commit] r2090 - in trunk: libexec
	src/org/griphyn/vdl/karajan/lib
Message-ID: <20080707161800.964ED9B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-07 11:17:57 -0500 (Mon, 07 Jul 2008)
New Revision: 2090

Added:
   trunk/src/org/griphyn/vdl/karajan/lib/WaitFieldValue.java
Modified:
   trunk/libexec/vdl-lib.xml
   trunk/libexec/vdl.k
Log:
WaitFieldValue helper

Modified: trunk/libexec/vdl-lib.xml
===================================================================
--- trunk/libexec/vdl-lib.xml	2008-07-07 12:39:21 UTC (rev 2089)
+++ trunk/libexec/vdl-lib.xml	2008-07-07 16:17:57 UTC (rev 2090)
@@ -45,6 +45,7 @@
 	<export name="getFieldSubscript"><elementDef classname="org.griphyn.vdl.karajan.lib.GetFieldSubscript"/></export>
 	<export name="setFieldValue"><elementDef classname="org.griphyn.vdl.karajan.lib.SetFieldValue"/></export>
 	<export name="getFieldValue"><elementDef classname="org.griphyn.vdl.karajan.lib.GetFieldValue"/></export>
+	<export name="waitFieldValue"><elementDef classname="org.griphyn.vdl.karajan.lib.WaitFieldValue"/></export>
 	<export name="isFileBound"><elementDef classname="org.griphyn.vdl.karajan.lib.IsFileBound"/></export>
 	<export name="fileSet"><elementDef classname="org.griphyn.vdl.karajan.lib.FileSet"/></export>
 	<export name="fringePaths"><elementDef classname="org.griphyn.vdl.karajan.lib.FringePaths"/></export>

Modified: trunk/libexec/vdl.k
===================================================================
--- trunk/libexec/vdl.k	2008-07-07 12:39:21 UTC (rev 2089)
+++ trunk/libexec/vdl.k	2008-07-07 16:17:57 UTC (rev 2090)
@@ -59,7 +59,7 @@
 						fp := vdl:fringePaths(var)
 						try (
 							for(path, fp
-								discard(vdl:getFieldValue(path=path, var))
+								vdl:waitFieldValue(path=path, var)
 							)
 							catch(".*errors in data dependencies.*"
 								log(LOG:DEBUG, exception)
@@ -80,7 +80,7 @@
 				)
 				else(
 					//we still wait until the primitive value is there
-					discard(vdl:getFieldValue(var))
+					vdl:waitFieldValue(var)
 				)
 			)
 		)

Added: trunk/src/org/griphyn/vdl/karajan/lib/WaitFieldValue.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/WaitFieldValue.java	                        (rev 0)
+++ trunk/src/org/griphyn/vdl/karajan/lib/WaitFieldValue.java	2008-07-07 16:17:57 UTC (rev 2090)
@@ -0,0 +1,57 @@
+package org.griphyn.vdl.karajan.lib;
+
+import java.util.Map;
+
+import org.apache.log4j.Logger;
+import org.globus.cog.karajan.arguments.Arg;
+import org.globus.cog.karajan.stack.VariableStack;
+import org.globus.cog.karajan.workflow.ExecutionException;
+import org.globus.cog.karajan.workflow.futures.FutureNotYetAvailable;
+import org.griphyn.vdl.karajan.PairIterator;
+import org.griphyn.vdl.mapping.DSHandle;
+import org.griphyn.vdl.mapping.HandleOpenException;
+import org.griphyn.vdl.mapping.InvalidPathException;
+import org.griphyn.vdl.mapping.Path;
+
+public class WaitFieldValue extends VDLFunction {
+	public static final Logger logger = Logger.getLogger(WaitFieldValue.class);
+
+	static {
+		setArguments(WaitFieldValue.class, new Arg[] { PA_VAR, OA_PATH });
+	}
+
+	/**
+	 * Takes a supplied variable and path, and returns the unique value at that
+	 * path. Path can contain wildcards, in which case an array is returned.
+	 */
+	public Object function(VariableStack stack) throws ExecutionException {
+		Object var1 = PA_VAR.getValue(stack);
+		if (!(var1 instanceof DSHandle)) {
+			throw new RuntimeException("Can only wait for DSHandles - was supplied "+var1.getClass());
+		}
+		DSHandle var = (DSHandle) var1;
+		try {
+			Path path = parsePath(OA_PATH.getValue(stack), stack);
+			var = var.getField(path);
+			synchronized (var) {
+				if (!var.isClosed()) {
+					logger.debug("Waiting for " + var);
+					throw new FutureNotYetAvailable(addFutureListener(stack, var));
+				}
+				else {
+					Object v = var.getValue();
+					logger.debug("Do not need to wait for " + var+" as it is closed and has value "+v + (v!=null ? " with class "+v.getClass() : "" ));
+					if(v !=null && v instanceof RuntimeException) {
+						throw (RuntimeException)v;
+					} else {
+						return null;
+					}
+				}
+			}
+		}
+		catch (InvalidPathException e) {
+			throw new ExecutionException(e);
+		}
+	}
+
+}



From noreply at svn.ci.uchicago.edu  Mon Jul  7 11:57:51 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 11:57:51 -0500 (CDT)
Subject: [Swift-commit] r2092 - in trunk: libexec
	src/org/griphyn/vdl/karajan/lib
Message-ID: <20080707165751.604E09B4002@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-07 11:57:50 -0500 (Mon, 07 Jul 2008)
New Revision: 2092

Modified:
   trunk/libexec/vdl-int.k
   trunk/src/org/griphyn/vdl/karajan/lib/LogVar.java
Log:
remove unused restart parameters

Modified: trunk/libexec/vdl-int.k
===================================================================
--- trunk/libexec/vdl-int.k	2008-07-07 16:55:08 UTC (rev 2091)
+++ trunk/libexec/vdl-int.k	2008-07-07 16:57:50 UTC (rev 2092)
@@ -246,7 +246,7 @@
 					    task:transfer(srchost=host, srcfile=bname,
 					        srcdir=rdir, destdir=ldir, desthost=dhost, destprovider=provider)
 					)
-					vdl:logvar(var, path, host, dir, bname)
+					vdl:logvar(var, path)
 					log(LOG:DEBUG, "FILE_STAGE_OUT_END srcname={bname} srcdir={rdir} srchost={host} ",
 						"destdir={ldir} desthost={dhost} provider={provider}")
 	

Modified: trunk/src/org/griphyn/vdl/karajan/lib/LogVar.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/LogVar.java	2008-07-07 16:55:08 UTC (rev 2091)
+++ trunk/src/org/griphyn/vdl/karajan/lib/LogVar.java	2008-07-07 16:57:50 UTC (rev 2092)
@@ -12,12 +12,9 @@
 import org.griphyn.vdl.mapping.Path;
 
 public class LogVar extends VDLFunction {
-	public static final Arg PA_HOST = new Arg.Positional("host");
-	public static final Arg PA_DIR = new Arg.Positional("dir");
-	public static final Arg PA_NAME = new Arg.Positional("name");
 
 	static {
-		setArguments(LogVar.class, new Arg[] { PA_VAR, PA_PATH, PA_HOST, PA_DIR, PA_NAME });
+		setArguments(LogVar.class, new Arg[] { PA_VAR, PA_PATH });
 	}
 
 	public Object function(VariableStack stack) throws ExecutionException {



From noreply at svn.ci.uchicago.edu  Mon Jul  7 12:21:04 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 12:21:04 -0500 (CDT)
Subject: [Swift-commit] r2093 - in trunk: libexec
	src/org/griphyn/vdl/karajan/lib src/org/griphyn/vdl/mapping
Message-ID: <20080707172104.B33709B4002@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-07 12:21:04 -0500 (Mon, 07 Jul 2008)
New Revision: 2093

Added:
   trunk/src/org/griphyn/vdl/karajan/lib/IsRestartable.java
Modified:
   trunk/libexec/execute-default.k
   trunk/libexec/execute-dryrun.k
   trunk/libexec/execute-typecheck.k
   trunk/libexec/vdl-int.k
   trunk/libexec/vdl-lib.xml
   trunk/libexec/vdl.k
   trunk/src/org/griphyn/vdl/mapping/AbstractDataNode.java
   trunk/src/org/griphyn/vdl/mapping/DSHandle.java
Log:
distinguish between file-mapped and restartable datasets, ready for non-file-based datasets

Modified: trunk/libexec/execute-default.k
===================================================================
--- trunk/libexec/execute-default.k	2008-07-07 16:57:50 UTC (rev 2092)
+++ trunk/libexec/execute-default.k	2008-07-07 17:21:04 UTC (rev 2093)
@@ -1,9 +1,9 @@
-element([tr, optional(arguments, stdin, stdout, stderr, deperror, mdeperror), channel(stagein), channel(stageout)]
+element([tr, optional(arguments, stdin, stdout, stderr, deperror, mdeperror), channel(stagein), channel(stageout), channel(restartout)]
 
 	vdl:initprogressstate()
 	vdl:setprogress("Initializing")
 
-	done := isDone(stageout)
+	done := isDone(restartout)
 	derr := try(deperror, false)
 	merr := try(mdeperror, false)
 
@@ -23,13 +23,13 @@
 									execute2(
 										tr, maybe(arguments=arguments), 
 										maybe(stdin=stdin), maybe(stdout=stdout), maybe(stderr=stderr), 
-										stagein, stageout, replicationGroup, replicationChannel
+										stagein, stageout, restartout, replicationGroup, replicationChannel
 									)
 									catch("^Abort$")
 								)
 							)
 						)
-						mark(stageout, err=false, mapping=false)
+						mark(restartout, err=false, mapping=false)
 						graphStuff(tr, stagein, stageout, err=false, maybe(args=arguments))
 						echo("{tr} completed")
 						log(LOG:INFO, "END_SUCCESS thread={#thread} tr={tr}")
@@ -46,7 +46,7 @@
 							else (
 								to(errors, exception)
 								log(LOG:INFO, exception)
-								mark(stageout, err=true, mapping=false)
+								mark(restartout, err=true, mapping=false)
 								graphStuff(tr, stagein, stageout, err=true, maybe(args=arguments))
 							)
 						)
@@ -68,7 +68,7 @@
 					log(LOG:INFO, exception)
 				)
 			)
-			mark(stageout, err=true, mapping=merr)
+			mark(restartout, err=true, mapping=merr)
 			graphStuff(tr, stagein, stageout, err=true, maybe(args=arguments))
 		)
 	)

Modified: trunk/libexec/execute-dryrun.k
===================================================================
--- trunk/libexec/execute-dryrun.k	2008-07-07 16:57:50 UTC (rev 2092)
+++ trunk/libexec/execute-dryrun.k	2008-07-07 17:21:04 UTC (rev 2093)
@@ -1,4 +1,4 @@
-element([tr, optional(arguments, stdin, stdout, stderr, deperror), channel(stagein), channel(stageout)]
+element([tr, optional(arguments, stdin, stdout, stderr, deperror), channel(stagein), channel(stageout), channel(restartout)]
 
 	done := isDone(stageout)
 				

Modified: trunk/libexec/execute-typecheck.k
===================================================================
--- trunk/libexec/execute-typecheck.k	2008-07-07 16:57:50 UTC (rev 2092)
+++ trunk/libexec/execute-typecheck.k	2008-07-07 17:21:04 UTC (rev 2093)
@@ -1,3 +1,3 @@
-element([tr, optional(arguments, stdin, stdout, stderr), channel(stagein), channel(stageout)]
+element([tr, optional(arguments, stdin, stdout, stderr), channel(stagein), channel(stageout), channel(restartout)]
 	mark(stageout, err=false)
 )

Modified: trunk/libexec/vdl-int.k
===================================================================
--- trunk/libexec/vdl-int.k	2008-07-07 16:57:50 UTC (rev 2092)
+++ trunk/libexec/vdl-int.k	2008-07-07 17:21:04 UTC (rev 2093)
@@ -41,9 +41,9 @@
 			)
 		)
 			
-		element(mark, [stageout, err, optional(mapping)]
+		element(mark, [restarts, err, optional(mapping)]
 			if(
-				err for(pv, stageout
+				err for(pv, restarts
 					[path, var] := each(pv)
 					vdl:setFutureFault(var, path=path, mapping=mapping)
 				)
@@ -246,7 +246,6 @@
 					    task:transfer(srchost=host, srcfile=bname,
 					        srcdir=rdir, destdir=ldir, desthost=dhost, destprovider=provider)
 					)
-					vdl:logvar(var, path)
 					log(LOG:DEBUG, "FILE_STAGE_OUT_END srcname={bname} srcdir={rdir} srchost={host} ",
 						"destdir={ldir} desthost={dhost} provider={provider}")
 	
@@ -261,6 +260,13 @@
 			)
 			log(LOG:INFO, "END jobid={jobid} - Staging out finished")
 		)
+
+		element(doRestartlog, [restartouts]
+			uParallelFor(f, restartouts,
+				[path, var] := each(f)
+				vdl:logvar(var, path)
+			)
+		)
 		
 		element(graphStuff, [tr, stagein, stageout, err, optional(args)]
 			if(
@@ -352,7 +358,7 @@
 			recfile
 		)
 
-		element(execute2, [tr, optional(arguments, stdin, stdout, stderr), stagein, stageout, 
+		element(execute2, [tr, optional(arguments, stdin, stdout, stderr), stagein, stageout,  restartout,
 			replicationGroup, replicationChannel]
 			stagein := list(unique(each(stagein)))
 			stageout := list(unique(each(stageout)))
@@ -418,6 +424,7 @@
 						
 						vdl:setprogress("Stage out")
 						doStageout(jobid, stageout, sharedDir, rhost)
+						doRestartlog(restartout)
 						if(
 							kickstart != "" & vdl:configProperty("kickstart.always.transfer") == "true"
 							discard(transferKickstartRec(rhost, wfdir, jobid, jobdir))

Modified: trunk/libexec/vdl-lib.xml
===================================================================
--- trunk/libexec/vdl-lib.xml	2008-07-07 16:57:50 UTC (rev 2092)
+++ trunk/libexec/vdl-lib.xml	2008-07-07 17:21:04 UTC (rev 2093)
@@ -47,6 +47,7 @@
 	<export name="getFieldValue"><elementDef classname="org.griphyn.vdl.karajan.lib.GetFieldValue"/></export>
 	<export name="waitFieldValue"><elementDef classname="org.griphyn.vdl.karajan.lib.WaitFieldValue"/></export>
 	<export name="isFileBound"><elementDef classname="org.griphyn.vdl.karajan.lib.IsFileBound"/></export>
+	<export name="isRestartable"><elementDef classname="org.griphyn.vdl.karajan.lib.IsRestartable"/></export>
 	<export name="fileSet"><elementDef classname="org.griphyn.vdl.karajan.lib.FileSet"/></export>
 	<export name="fringePaths"><elementDef classname="org.griphyn.vdl.karajan.lib.FringePaths"/></export>
 	<export name="assign"><elementDef classname="org.griphyn.vdl.karajan.lib.Assign"/></export>

Modified: trunk/libexec/vdl.k
===================================================================
--- trunk/libexec/vdl.k	2008-07-07 16:57:50 UTC (rev 2092)
+++ trunk/libexec/vdl.k	2008-07-07 17:21:04 UTC (rev 2093)
@@ -100,6 +100,20 @@
 					deperror = true
 				)
 			)
+			try(
+				if(vdl:isRestartable(var) 
+					channel:to(restartout,
+						for(path, vdl:fringePaths(var)
+							list(path, var)
+						)
+					)
+				)
+				catch(".*not mapped.*"
+					log(LOG:DEBUG, exception)
+					mdeperror = true
+					deperror = true
+				)
+			)
 		)
 
 		element(mapping, [descriptor, ...]

Added: trunk/src/org/griphyn/vdl/karajan/lib/IsRestartable.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/IsRestartable.java	                        (rev 0)
+++ trunk/src/org/griphyn/vdl/karajan/lib/IsRestartable.java	2008-07-07 17:21:04 UTC (rev 2093)
@@ -0,0 +1,25 @@
+package org.griphyn.vdl.karajan.lib;
+
+import org.globus.cog.karajan.arguments.Arg;
+import org.globus.cog.karajan.stack.VariableStack;
+import org.globus.cog.karajan.workflow.ExecutionException;
+import org.griphyn.vdl.mapping.AbstractDataNode;
+import org.griphyn.vdl.mapping.DSHandle;
+
+/** Determines if a variable is 'restartable'; that is, if we restart the
+    workflow, will this variable still have its content.
+*/
+    
+    
+
+public class IsRestartable extends VDLFunction {
+	static {
+		setArguments(IsRestartable.class, new Arg[] { PA_VAR });
+	}
+
+	public Object function(VariableStack stack) throws ExecutionException {
+		DSHandle var = (DSHandle) PA_VAR.getValue(stack);
+		return Boolean.valueOf(var.isRestartable());
+	}
+}
+

Modified: trunk/src/org/griphyn/vdl/mapping/AbstractDataNode.java
===================================================================
--- trunk/src/org/griphyn/vdl/mapping/AbstractDataNode.java	2008-07-07 16:57:50 UTC (rev 2092)
+++ trunk/src/org/griphyn/vdl/mapping/AbstractDataNode.java	2008-07-07 17:21:04 UTC (rev 2093)
@@ -65,6 +65,10 @@
 		return field.getType().isPrimitive();
 	}
 
+	public boolean isRestartable() {
+		return !isPrimitive();
+	}
+
 	protected Field getField() {
 		return field;
 	}

Modified: trunk/src/org/griphyn/vdl/mapping/DSHandle.java
===================================================================
--- trunk/src/org/griphyn/vdl/mapping/DSHandle.java	2008-07-07 16:57:50 UTC (rev 2092)
+++ trunk/src/org/griphyn/vdl/mapping/DSHandle.java	2008-07-07 17:21:04 UTC (rev 2093)
@@ -82,4 +82,6 @@
     Mapper getMapper();
 
     public String getIdentifier();
+
+    public boolean isRestartable();
 }



From noreply at svn.ci.uchicago.edu  Mon Jul  7 11:55:08 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 11:55:08 -0500 (CDT)
Subject: [Swift-commit] r2091 - trunk/src/org/griphyn/vdl/karajan/lib
Message-ID: <20080707165508.BE2CE9B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-07 11:55:08 -0500 (Mon, 07 Jul 2008)
New Revision: 2091

Modified:
   trunk/src/org/griphyn/vdl/karajan/lib/IsLogged.java
   trunk/src/org/griphyn/vdl/karajan/lib/LogVar.java
   trunk/src/org/griphyn/vdl/karajan/lib/New.java
Log:
restarts based on variable name rather than filename

Modified: trunk/src/org/griphyn/vdl/karajan/lib/IsLogged.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/IsLogged.java	2008-07-07 16:17:57 UTC (rev 2090)
+++ trunk/src/org/griphyn/vdl/karajan/lib/IsLogged.java	2008-07-07 16:55:08 UTC (rev 2091)
@@ -3,6 +3,7 @@
  */
 package org.griphyn.vdl.karajan.lib;
 
+import java.util.List;
 import java.util.Map;
 
 import org.globus.cog.karajan.arguments.Arg;
@@ -10,7 +11,6 @@
 import org.globus.cog.karajan.util.TypeUtil;
 import org.globus.cog.karajan.workflow.ExecutionException;
 import org.globus.cog.karajan.workflow.nodes.restartLog.LogEntry;
-import org.globus.cog.karajan.workflow.nodes.restartLog.MutableInteger;
 import org.griphyn.vdl.mapping.DSHandle;
 import org.griphyn.vdl.mapping.Path;
 
@@ -30,13 +30,12 @@
 			path = Path.parse(TypeUtil.toString(p));
 		}
 		path = var.getPathFromRoot().append(path);
-		String file = var.getMapper().map(path).toString();
-		LogEntry entry = LogEntry.build(file);
+		LogEntry entry = LogEntry.build(var.getRoot().getParam("swift#restartid") + "." + path.stringForm());
 		Map map = getLogData(stack);
 		boolean found = false;
 		synchronized (map) {
-			MutableInteger count = (MutableInteger) map.get(entry);
-			if (count != null && count.getValue() > 0) {
+			List files = (List) map.get(entry);
+			if (files != null && !files.isEmpty()) {
 				found = true;
 			}
 		}

Modified: trunk/src/org/griphyn/vdl/karajan/lib/LogVar.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/LogVar.java	2008-07-07 16:17:57 UTC (rev 2090)
+++ trunk/src/org/griphyn/vdl/karajan/lib/LogVar.java	2008-07-07 16:55:08 UTC (rev 2091)
@@ -31,7 +31,14 @@
             path = Path.parse(TypeUtil.toString(p));
         }
 		path = var.getPathFromRoot().append(path);
-		RestartLog.LOG_CHANNEL.ret(stack, var.getMapper().map(path).toString());
+		String annotation;
+		if(var.getMapper() != null) {
+			annotation = "" + var.getMapper().map(path);
+		} else {
+			annotation = "unmapped";
+		}
+		RestartLog.LOG_CHANNEL.ret(stack, var.getRoot().getParam("swift#restartid")
+				+ "." + path.stringForm() + "!" + annotation);
 		return null;
 	}
 }

Modified: trunk/src/org/griphyn/vdl/karajan/lib/New.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/New.java	2008-07-07 16:17:57 UTC (rev 2090)
+++ trunk/src/org/griphyn/vdl/karajan/lib/New.java	2008-07-07 16:55:08 UTC (rev 2091)
@@ -52,6 +52,8 @@
 			mapping.put("dbgname", dbgname);
 		}
 
+		mapping.put("swift#restartid", getThreadPrefix(stack) + ":" + dbgname);
+
 		if(waitfor != null) {
 			mapping.put("waitfor", waitfor);
 		}



From noreply at svn.ci.uchicago.edu  Mon Jul  7 14:37:14 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon,  7 Jul 2008 14:37:14 -0500 (CDT)
Subject: [Swift-commit] r2094 - SwiftApps/SIDGrid/config
Message-ID: <20080707193714.4500B9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-07 14:37:13 -0500 (Mon, 07 Jul 2008)
New Revision: 2094

Modified:
   SwiftApps/SIDGrid/config/tc.data
Log:
updated to ucteraport scripts location

Modified: SwiftApps/SIDGrid/config/tc.data
===================================================================
--- SwiftApps/SIDGrid/config/tc.data	2008-07-07 17:21:04 UTC (rev 2093)
+++ SwiftApps/SIDGrid/config/tc.data	2008-07-07 19:37:13 UTC (rev 2094)
@@ -869,8 +869,8 @@
 UCTERAPORT   whirlgif   /gpfs1/osg_data/SIDGrid/usr/bin/whirlgif   INSTALLED INTEL64::LINUX null
 UCTERAPORT   Xphace   /gpfs1/osg_data/SIDGrid/usr/bin/Xphace   INSTALLED INTEL64::LINUX null
 #-------------------------------------------------------
-UCTERAPORT	RInvoke		/gpfs1/osg_data/sidgrid_tools/scripts/RInvoke.sh   INSTALLED INTEL64::LINUX GLOBUS::maxwalltime=300,queue=short
-UCTERAPORT	Mediator	/gpfs1/osg_data/sidgrid_tools/scripts/Mediator.py INSTALLED INTEL64::LINUX env::LD_LIBRARY_PATH=/gpfs1/osg_data/SIDGrid/usr/lib
+UCTERAPORT	RInvoke		/gpfs1/osg_data/SIDGrid/scripts/RInvoke.sh   INSTALLED INTEL64::LINUX GLOBUS::maxwalltime=300,queue=short
+UCTERAPORT	Mediator	/gpfs1/osg_data/SIDGrid/scripts/Mediator.py INSTALLED INTEL64::LINUX env::LD_LIBRARY_PATH=/gpfs1/osg_data/SIDGrid/usr/lib
 
 ####-----------------NCSATUNG-----------------------------
 #------AFNI
@@ -1448,3 +1448,12 @@
 ANLUCTERAGRID64   Mediator /soft/community/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX  null
 ANLUCTERAGRID64   echo /bin/echo  INSTALLED INTEL64::LINUX  null
 ANLUCTERAGRID64   RInvoke /soft/community/SIDGrid/scripts/RInvoke.sh  INSTALLED INTEL64::LINUX  null
+ANLUCTERAGRID64   env /bin/env INSTALLED INTEL64::LINUX  null
+
+####----------------RANGER---------------------------------------
+
+RANGER RInvoke /scratch/projects/tg/SIDGrid/scripts/RInvoke.sh  INSTALLED INTEL64::LINUX   env::PATHPREFIX=/scratch/projects/tg/SIDGrid/usr/bin
+RANGER echo /bin/echo  INSTALLED INTEL64::LINUX   null
+RANGER env /usr/bin/env  INSTALLED INTEL64::LINUX   null
+
+



From noreply at svn.ci.uchicago.edu  Tue Jul  8 01:55:32 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue,  8 Jul 2008 01:55:32 -0500 (CDT)
Subject: [Swift-commit] r2095 - in trunk: src/org/griphyn/vdl/karajan/lib
	src/org/griphyn/vdl/mapping src/org/griphyn/vdl/type
	tests/language-behaviour tests/misc
Message-ID: <20080708065533.0AE209B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-08 01:55:32 -0500 (Tue, 08 Jul 2008)
New Revision: 2095

Added:
   trunk/src/org/griphyn/vdl/mapping/ExternalDataNode.java
   trunk/tests/language-behaviour/087-external-dataset.swift
   trunk/tests/misc/ordering-extern.sh
   trunk/tests/misc/restart-extern.sh
   trunk/tests/misc/restart-extern.swift
   trunk/tests/misc/restart-external.in
Modified:
   trunk/src/org/griphyn/vdl/karajan/lib/New.java
   trunk/src/org/griphyn/vdl/type/Types.java
   trunk/tests/misc/run
Log:
externally stored datasets

Modified: trunk/src/org/griphyn/vdl/karajan/lib/New.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/New.java	2008-07-07 19:37:13 UTC (rev 2094)
+++ trunk/src/org/griphyn/vdl/karajan/lib/New.java	2008-07-08 06:55:32 UTC (rev 2095)
@@ -18,6 +18,7 @@
 import org.griphyn.vdl.mapping.Path;
 import org.griphyn.vdl.mapping.RootArrayDataNode;
 import org.griphyn.vdl.mapping.RootDataNode;
+import org.griphyn.vdl.mapping.ExternalDataNode;
 import org.griphyn.vdl.mapping.file.ConcurrentMapper;
 import org.griphyn.vdl.type.Type;
 import org.griphyn.vdl.type.Types;
@@ -79,7 +80,9 @@
 				type = Types.getType(typename);
 			}
 			DSHandle handle;
-			if (type.isArray()) {
+			if(typename.equals("external")) {
+				handle = new ExternalDataNode();
+			} else if (type.isArray()) {
 				// dealing with array variable
 				handle = new RootArrayDataNode(type);
 				if (value != null) {

Added: trunk/src/org/griphyn/vdl/mapping/ExternalDataNode.java
===================================================================
--- trunk/src/org/griphyn/vdl/mapping/ExternalDataNode.java	                        (rev 0)
+++ trunk/src/org/griphyn/vdl/mapping/ExternalDataNode.java	2008-07-08 06:55:32 UTC (rev 2095)
@@ -0,0 +1,299 @@
+package org.griphyn.vdl.mapping;
+
+import org.griphyn.vdl.karajan.Loader;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.log4j.Logger;
+import org.griphyn.vdl.type.Field;
+import org.griphyn.vdl.type.NoSuchTypeException;
+import org.griphyn.vdl.type.Type;
+import org.griphyn.vdl.type.Types;
+
+public class ExternalDataNode implements DSHandle {
+
+        private Map params;
+
+        public void init(Map params) {
+                this.params = params;
+	}
+
+	static final String DATASET_URI_PREFIX = "tag:benc at ci.uchicago.edu,2008:swift:dataset:external:";
+
+	public static final Logger logger = Logger.getLogger(ExternalDataNode.class);
+	
+	public static final MappingParam PARAM_PREFIX = new MappingParam("prefix", null);
+
+	/** Datasets are identified within a run by this sequence number and the
+	    partial ID field.
+	    The initial value is chosen to aid human recognition of sequence
+	    numbers in the wild. There is no requirement that it start at this
+	    (or any other) particular value. Note that this introduces a
+	    maximum on the number of datasets which can be dealt with in any
+	    run to be about 2^62. */
+	private static long datasetIDCounter = 720000000000l;
+
+	/** This is used to provide a (hopefully) globally unique identifier for
+	    each time the datasetIDCounter is reset (whenever this class is
+	    loaded, which will usually happen once per JVM). No meaning should be
+	    inferred from this value - it exists purely for making unique URIs. */
+	private static final String datasetIDPartialID = Loader.getUUID();
+
+	private Map handles;
+	private Object value;
+	private boolean closed;
+	private List listeners;
+	final String identifierURI = makeIdentifierURIString();
+
+	public ExternalDataNode() {
+	}
+
+	public Type getType() {
+		try {
+			return Types.getType("external");
+		} catch(NoSuchTypeException te) {
+			throw new RuntimeException(te);
+		}
+	}
+
+	public boolean isPrimitive() {
+		return false;
+	}
+
+	public boolean isRestartable() {
+		return true;
+	}
+
+	/**
+	 * create a String representation of this node. If the node has a value,
+	 * then uses the String representation of that value. Otherwise, generates a
+	 * text description.
+	 */
+	public String toString() {
+		if (this.value != null && !(this.value instanceof Exception)) {
+			// special handling for ints...
+			if (this.getType().equals(Types.INT)) {
+				try {
+					Number n = (Number) this.getValue();
+					return String.valueOf(n.intValue());
+				}
+				catch (ClassCastException e) {
+					throw new RuntimeException("Internal type error. Value is not a Number for "
+							+ getDisplayableName() + getPathFromRoot());
+				}
+			}
+			else {
+				return this.value.toString();
+			}
+		}
+
+		String prefix = this.getClass().getName();
+
+		prefix = prefix + " identifier "+this.getIdentifier(); 
+
+ 		prefix = prefix + " with no value at dataset=";
+
+		prefix = prefix + getDisplayableName();
+
+		if (!Path.EMPTY_PATH.equals(getPathFromRoot())) {
+			prefix = prefix + " path="+ getPathFromRoot().toString();
+		}
+
+		if(closed) {
+			prefix = prefix + " (closed)";
+		}
+		else {
+			prefix = prefix + " (not closed)";
+		}
+
+		return prefix;
+	}
+
+	public DSHandle getRoot() {
+		return this;
+	}
+
+	protected String getDisplayableName() {
+		String prefix = getRoot().getParam("dbgname");
+		if (prefix == null) {
+			prefix = getRoot().getParam("prefix");
+		}
+		if (prefix == null) {
+			prefix = "unnamed SwiftScript value";
+		}
+		return prefix;
+	}
+
+	public DSHandle getField(Path path) throws InvalidPathException {
+		if (path.isEmpty()) {
+			return this;
+		} else {
+			throw new InvalidPathException(path, this);
+		}
+	}
+
+	public Collection getFields(Path path) throws InvalidPathException, HandleOpenException {
+		List fields = new ArrayList();
+		return fields;
+	}
+
+	public void set(DSHandle handle) {
+		throw new IllegalArgumentException(this.getDisplayableName() + " is an external dataset and cannot be set");
+	}
+
+	protected void setField(String name, DSHandle handle) {
+		synchronized (handles) {
+			handles.put(name, handle);
+		}
+	}
+
+	protected DSHandle getHandle(String name) {
+		synchronized (handles) {
+			return (DSHandle) handles.get(name);
+		}
+	}
+
+	protected boolean isHandlesEmpty() {
+		synchronized (handles) {
+			return handles.isEmpty();
+		}
+	}
+
+	protected void checkDataException() {
+		if (value instanceof DependentException) {
+			throw (DependentException) value;
+		}
+	}
+
+	protected void checkMappingException() {
+		if (value instanceof MappingDependentException) {
+			throw (MappingDependentException) value;
+		}
+	}
+
+	public Object getValue() {
+logger.warn("getValue called in an external dataset");
+return value;
+// throw new RuntimeException("cannot get value of external dataset");
+	}
+
+	public Map getArrayValue() {
+throw new RuntimeException("cannot get value of external dataset");
+	}
+
+	public boolean isArray() {
+		return false;
+	}
+
+	public void setValue(Object value) {
+		if (this.closed) {
+			throw new IllegalArgumentException(this.getDisplayableName()
+					+ " is closed with a value of "+this.value);
+		}
+		if (this.value != null) {
+			throw new IllegalArgumentException(this.getDisplayableName()
+					+ " is already assigned with a value of " + this.value);
+		}
+		this.value = value;
+	}
+
+	public Collection getFringePaths() throws HandleOpenException {
+		ArrayList list = new ArrayList();
+		list.add(Path.EMPTY_PATH);
+		return list;
+	}
+
+	public synchronized void closeShallow() {
+		this.closed = true;
+		notifyListeners();
+		logger.info("closed "+this.getIdentifier());
+	}
+
+	public boolean isClosed() {
+		return closed;
+	}
+
+	public void closeDeep() {
+		if (!this.closed) {
+			closeShallow();
+		}
+		synchronized (handles) {
+			Iterator i = handles.entrySet().iterator();
+			while (i.hasNext()) {
+				Map.Entry e = (Map.Entry) i.next();
+				AbstractDataNode mapper = (AbstractDataNode) e.getValue();
+				mapper.closeDeep();
+			}
+		}
+	}
+
+	public Path getPathFromRoot() {
+		return Path.EMPTY_PATH;
+	}
+
+	public Mapper getMapper() {
+		return null;
+	}
+
+	protected Map getHandles() {
+		return handles;
+	}
+
+	public synchronized void addListener(DSHandleListener listener) {
+		if (logger.isInfoEnabled()) {
+			logger.info("Adding handle listener \"" + listener + "\" to \"" + this + "\"");
+		}
+		if (listeners == null) {
+			listeners = new LinkedList();
+		}
+		listeners.add(listener);
+		if (closed) {
+			notifyListeners();
+		}
+	}
+
+	protected synchronized void notifyListeners() {
+		if (listeners != null) {
+			Iterator i = listeners.iterator();
+			while (i.hasNext()) {
+				DSHandleListener listener = (DSHandleListener) i.next();
+				i.remove();
+				if (logger.isInfoEnabled()) {
+					logger.info("Notifying listener \"" + listener + "\" about \"" + this + "\"");
+				}
+				listener.handleClosed(this);
+			}
+			listeners = null;
+		}
+	}
+
+	public String getIdentifier() {
+		return identifierURI;
+	}
+
+	String makeIdentifierURIString() {
+		datasetIDCounter++;
+		return DATASET_URI_PREFIX + datasetIDPartialID + ":" + datasetIDCounter; 
+	}
+
+        public String getParam(String name) {
+                if (params == null) {
+                        return null;
+                }
+                return (String) params.get(name);
+        }
+
+  public DSHandle createDSHandle(String fieldName) {
+throw new RuntimeException("cannot create new field in external dataset");
+                }
+
+        public DSHandle getParent() {
+                return null;
+        }
+}

Modified: trunk/src/org/griphyn/vdl/type/Types.java
===================================================================
--- trunk/src/org/griphyn/vdl/type/Types.java	2008-07-07 19:37:13 UTC (rev 2094)
+++ trunk/src/org/griphyn/vdl/type/Types.java	2008-07-08 06:55:32 UTC (rev 2095)
@@ -55,7 +55,7 @@
 		return type;
 	}
 
-	public static final Type INT, STRING, FLOAT, BOOLEAN, ANY;
+	public static final Type INT, STRING, FLOAT, BOOLEAN, ANY, EXTERNAL;
 
 	// add built-in primitive types
 	static {
@@ -64,6 +64,7 @@
 		FLOAT = addPrimitiveType("float");
 		BOOLEAN = addPrimitiveType("boolean");
 		ANY = addPrimitiveType("any");
+		EXTERNAL = addPrimitiveType("external");
 	}
 
 	public synchronized static void resolveTypes() throws NoSuchTypeException {

Added: trunk/tests/language-behaviour/087-external-dataset.swift
===================================================================
--- trunk/tests/language-behaviour/087-external-dataset.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/087-external-dataset.swift	2008-07-08 06:55:32 UTC (rev 2095)
@@ -0,0 +1,19 @@
+type file;
+
+(external o) a() {
+    app {
+        touch "foo";
+    }
+}
+
+b(external o) {
+    app {
+        touch "bar";
+    }
+}
+
+external sync;
+
+sync=a();
+b(sync);
+

Added: trunk/tests/misc/ordering-extern.sh
===================================================================
--- trunk/tests/misc/ordering-extern.sh	                        (rev 0)
+++ trunk/tests/misc/ordering-extern.sh	2008-07-08 06:55:32 UTC (rev 2095)
@@ -0,0 +1,51 @@
+#!/bin/bash
+
+export CF=swift.properties.restart-extern
+cat $(dirname $(which swift))/../etc/swift.properties | grep --invert-match -E '^lazy.errors=' > $CF
+echo lazy.errors=true >> $CF
+
+rm -f *.rlog restart-*.out restart-extern.kml restart-extern.xml restart-*.out
+
+rm -rf _concurrent
+
+echo "localhost	helperA	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" > tmp.restartOK.tc.data
+echo "localhost	helperB	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartOK.tc.data
+echo "localhost	helperC	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartOK.tc.data
+
+swift -config $CF -tc.file tmp.restartOK.tc.data restart-extern.swift -dir=`pwd`
+
+PRECHECKEXIT=$?
+
+
+if [ "$PRECHECKEXIT" != 0 ]; then
+  echo Failed - attempt to run workflow without ordering configuration failed
+  exit 1
+fi
+
+rm -f *.rlog restart-*.out restart-extern.kml restart-extern.xml
+rm -rf _concurrent
+
+# make A fail and B succeed.
+# extern dependency ordering should mean that B does not run.
+
+echo "localhost	helperA	$(pwd)/restart5-helper-fail	INSTALLED	INTEL32::LINUX	null" > tmp.restartB.tc.data
+echo "localhost	helperB	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartB.tc.data
+echo "localhost	helperC	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartB.tc.data
+
+swift -config $CF -tc.file tmp.restartB.tc.data restart-extern.swift -dir=`pwd`
+
+SECONDEXIT=$?
+
+if [ "$SECONDEXIT" = "0" ]; then
+  echo Failed - broken apps succeeded 
+  exit 2
+fi
+
+if [ -f restart-extern.2.out ] || [ -f restart-extern.1.out ]; then
+  echo Failed - output files came into existence that indicate external dependency was ignored
+  exit 3
+fi
+
+echo restart-exterm success
+exit 0
+

Added: trunk/tests/misc/restart-extern.sh
===================================================================
--- trunk/tests/misc/restart-extern.sh	                        (rev 0)
+++ trunk/tests/misc/restart-extern.sh	2008-07-08 06:55:32 UTC (rev 2095)
@@ -0,0 +1,64 @@
+#!/bin/bash
+
+export CF=swift.properties.restart5
+cat $(dirname $(which swift))/../etc/swift.properties | grep --invert-match -E '^lazy.errors=' > $CF
+echo lazy.errors=true >> $CF
+
+rm -f *.rlog restart-*.out restart5.kml restart5.xml restart5.*.out restart5.*.out.single-run
+
+rm -rf _concurrent
+
+echo "localhost	helperA	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" > tmp.restartOK.tc.data
+echo "localhost	helperB	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartOK.tc.data
+echo "localhost	helperC	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartOK.tc.data
+
+swift -config $CF -tc.file tmp.restartOK.tc.data restart-extern.swift -dir=`pwd`
+
+PRECHECKEXIT=$?
+
+
+if [ "$PRECHECKEXIT" != 0 ]; then
+  echo Failed - attempt to run workflow without restart configuration failed
+  exit 1
+fi
+
+rm -f *.rlog restart-*.out restart-extern.kml restart-extern.xml
+rm -rf _concurrent
+
+echo "localhost	helperA	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" > tmp.restartA.tc.data
+echo "localhost	helperB	$(pwd)/restart5-helper-fail	INSTALLED	INTEL32::LINUX	null" >> tmp.restartA.tc.data
+echo "localhost	helperC	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartA.tc.data
+
+swift -config $CF -tc.file tmp.restartA.tc.data restart-extern.swift -dir=`pwd`
+
+FIRSTEXIT=$?
+
+# this invocation should fail, with restart-1.out in existence but
+# not the others
+
+if [ "$FIRSTEXIT" == 0 ]; then
+  echo Failed - workflow was indicated as successfully completed the first time round.
+  exit 2
+fi
+
+# now make A fail - we should have run it already, and so we want to make
+# sure it does not run again; and make B succeed this time round.
+
+echo "localhost	helperA	$(pwd)/restart5-helper-fail	INSTALLED	INTEL32::LINUX	null" > tmp.restartB.tc.data
+echo "localhost	helperB	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartB.tc.data
+echo "localhost	helperC	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartB.tc.data
+
+# there should be only a single rlog here, because we deleted them all
+# at the start of this script.
+swift -config $CF -resume *.rlog -tc.file tmp.restartB.tc.data restart-extern.swift -dir=`pwd`
+
+SECONDEXIT=$?
+
+if [ "$SECONDEXIT" != "0" ]; then
+  echo Failed - second round failed
+  exit 3
+fi
+
+echo restart-exterm success
+exit 0
+

Added: trunk/tests/misc/restart-extern.swift
===================================================================
--- trunk/tests/misc/restart-extern.swift	                        (rev 0)
+++ trunk/tests/misc/restart-extern.swift	2008-07-08 06:55:32 UTC (rev 2095)
@@ -0,0 +1,18 @@
+type file;
+
+(external o) a() {
+    app {
+        helperA @strcat(@arg("dir"),"/restart-extern.1.out") "/etc/group" "qux";
+    }
+}
+
+b(external o) {
+    app {
+        helperB @strcat(@arg("dir"),"/restart-extern.2.out") "/etc/group" "baz";
+    }
+}
+
+external sync;
+
+sync=a();
+b(sync);

Added: trunk/tests/misc/restart-external.in
===================================================================
--- trunk/tests/misc/restart-external.in	                        (rev 0)
+++ trunk/tests/misc/restart-external.in	2008-07-08 06:55:32 UTC (rev 2095)
@@ -0,0 +1 @@
+foo

Modified: trunk/tests/misc/run
===================================================================
--- trunk/tests/misc/run	2008-07-07 19:37:13 UTC (rev 2094)
+++ trunk/tests/misc/run	2008-07-08 06:55:32 UTC (rev 2095)
@@ -1,5 +1,7 @@
 #!/bin/sh
-for a in clusters no-retries dryrun typecheck path-prefix restart restart2 restart3 restart4 restart5 restart-iterate workernode-local; do
+for a in clusters no-retries dryrun typecheck path-prefix restart restart2 restart3 restart4 restart5 restart-iterate workernode-local \
+ordering-extern-notlazy restart-extern ordering-extern \
+; do
  ./${a}.sh
  R=$?
  echo test $a ended with return value $R



From noreply at svn.ci.uchicago.edu  Tue Jul  8 03:27:05 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue,  8 Jul 2008 03:27:05 -0500 (CDT)
Subject: [Swift-commit] r2096 - trunk/tests/misc
Message-ID: <20080708082705.D9D3E9B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-08 03:27:05 -0500 (Tue, 08 Jul 2008)
New Revision: 2096

Added:
   trunk/tests/misc/ordering-extern-notlazy.sh
Log:
forgot to commit test

Added: trunk/tests/misc/ordering-extern-notlazy.sh
===================================================================
--- trunk/tests/misc/ordering-extern-notlazy.sh	                        (rev 0)
+++ trunk/tests/misc/ordering-extern-notlazy.sh	2008-07-08 08:27:05 UTC (rev 2096)
@@ -0,0 +1,51 @@
+#!/bin/bash
+
+export CF=swift.properties.restart-extern
+cat $(dirname $(which swift))/../etc/swift.properties | grep --invert-match -E '^lazy.errors=' > $CF
+echo lazy.errors=false >> $CF
+
+rm -f *.rlog restart-*.out restart-extern.kml restart-extern.xml restart-*.out
+
+rm -rf _concurrent
+
+echo "localhost	helperA	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" > tmp.restartOK.tc.data
+echo "localhost	helperB	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartOK.tc.data
+echo "localhost	helperC	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartOK.tc.data
+
+swift -config $CF -tc.file tmp.restartOK.tc.data restart-extern.swift -dir=`pwd`
+
+PRECHECKEXIT=$?
+
+
+if [ "$PRECHECKEXIT" != 0 ]; then
+  echo Failed - attempt to run workflow without ordering configuration failed
+  exit 1
+fi
+
+rm -f *.rlog restart-*.out restart-extern.kml restart-extern.xml
+rm -rf _concurrent
+
+# make A fail and B succeed.
+# extern dependency ordering should mean that B does not run.
+
+echo "localhost	helperA	$(pwd)/restart5-helper-fail	INSTALLED	INTEL32::LINUX	null" > tmp.restartB.tc.data
+echo "localhost	helperB	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartB.tc.data
+echo "localhost	helperC	$(pwd)/restart5-helper-success	INSTALLED	INTEL32::LINUX	null" >> tmp.restartB.tc.data
+
+swift -config $CF -tc.file tmp.restartB.tc.data restart-extern.swift -dir=`pwd`
+
+SECONDEXIT=$?
+
+if [ "$SECONDEXIT" = "0" ]; then
+  echo Failed - broken apps succeeded 
+  exit 2
+fi
+
+if [ -f restart-extern.2.out ] || [ -f restart-extern.1.out ]; then
+  echo Failed - output files came into existence that indicate external dependency was ignored
+  exit 3
+fi
+
+echo restart-exterm success
+exit 0
+


Property changes on: trunk/tests/misc/ordering-extern-notlazy.sh
___________________________________________________________________
Name: svn:executable
   + *



From noreply at svn.ci.uchicago.edu  Tue Jul  8 05:10:46 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue,  8 Jul 2008 05:10:46 -0500 (CDT)
Subject: [Swift-commit] r2097 - trunk/tests/misc
Message-ID: <20080708101046.2C0D49B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-08 05:10:45 -0500 (Tue, 08 Jul 2008)
New Revision: 2097

Modified:
   trunk/tests/misc/ordering-extern.sh
   trunk/tests/misc/restart-extern.sh
Log:
executable property on misc tests


Property changes on: trunk/tests/misc/ordering-extern.sh
___________________________________________________________________
Name: svn:executable
   + *


Property changes on: trunk/tests/misc/restart-extern.sh
___________________________________________________________________
Name: svn:executable
   + *



From noreply at svn.ci.uchicago.edu  Tue Jul  8 13:32:07 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue,  8 Jul 2008 13:32:07 -0500 (CDT)
Subject: [Swift-commit] r2098 - www
Message-ID: <20080708183207.0FD759B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-08 13:32:06 -0500 (Tue, 08 Jul 2008)
New Revision: 2098

Modified:
   www/index.php
Log:
funding and membership boilerplate

Modified: www/index.php
===================================================================
--- www/index.php	2008-07-08 10:10:45 UTC (rev 2097)
+++ www/index.php	2008-07-08 18:32:06 UTC (rev 2098)
@@ -69,6 +69,15 @@
 <div class="caption">"Swift applied to computational neuroscience. On the left, a small workflow from functional MRI study in 	aphasia; on the right, a map of brain activation clusters caused by various stimuli." 
 </div>
 
+<p>
+The Swift project is supported by the National Science Foundation with          
+additional support from NIH, Argonne National Laboratory and the University     
+of Chicago Computation Institute.
+</p>
+<p>
+Swift is an effort undergoing incubation at Globus. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful Globus projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by Globus.
+</p>
+
 				</div>
 				<!-- end left side content -->
 				<!-- right side content -->



From noreply at svn.ci.uchicago.edu  Wed Jul  9 12:48:35 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 12:48:35 -0500 (CDT)
Subject: [Swift-commit] r2099 - /
Message-ID: <20080709174835.3B3109B4001@www.ci.uchicago.edu>

Author: houzx
Date: 2008-07-09 12:48:33 -0500 (Wed, 09 Jul 2008)
New Revision: 2099

Removed:
   gasateria/
Log:
delete for change



From noreply at svn.ci.uchicago.edu  Wed Jul  9 12:51:21 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 12:51:21 -0500 (CDT)
Subject: [Swift-commit] r2100 - /
Message-ID: <20080709175121.CE0AC9B4001@www.ci.uchicago.edu>

Author: houzx
Date: 2008-07-09 12:51:19 -0500 (Wed, 09 Jul 2008)
New Revision: 2100

Added:
   adem/
Log:
New dir



From noreply at svn.ci.uchicago.edu  Wed Jul  9 13:18:49 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 13:18:49 -0500 (CDT)
Subject: [Swift-commit] r2102 - SwiftApps/SIDGrid/config
Message-ID: <20080709181849.5D4019B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-09 13:18:48 -0500 (Wed, 09 Jul 2008)
New Revision: 2102

Removed:
   SwiftApps/SIDGrid/config/sites_tung.xml
Log:
tungsten retired

Deleted: SwiftApps/SIDGrid/config/sites_tung.xml
===================================================================
--- SwiftApps/SIDGrid/config/sites_tung.xml	2008-07-09 17:52:40 UTC (rev 2101)
+++ SwiftApps/SIDGrid/config/sites_tung.xml	2008-07-09 18:18:48 UTC (rev 2102)
@@ -1,16 +0,0 @@
-<?xml version="1.0" encoding="ISO-8859-1"?>
-<config xmlns="http://www.griphyn.org/chimera/GVDS-PoolConfig" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.griphyn.org/chimera/GVDS-PoolConfig http://www.griphyn.org/chimera/sc-1.4.xsd" version="1.4">
- <import file="sys.xml"/>
-<import file="vdl-lib.xml"/>
-<set name="username">
-  <arg><vdl:new type="string" value="user"/></arg>
-</set>
- <!-- NCSATUNG @ grid-w.ncsa.teragrid.org -->
-  <pool handle="NCSATUNG" sysinfo="INTEL32::LINUX">
-    <profile namespace="env" key="data">/cfs/projects/sidgrid/sidgrid_out</profile>
-    <profile namespace="env" key="tmp">/tmp</profile>
-    <gridftp url="gsiftp://gt2-gridftp-w.ncsa.teragrid.org" storage="/cfs/projects/sidgrid_out" major="2" minor="4" patch="3"/>
-    <jobmanager universe="vanilla" url="grid-w.ncsa.teragrid.org/jobmanager-lsf" major="2" minor="4" patch="3"/>
-    <workdirectory>/cfs/projects/sidgrid/sidgrid_out/{username}</workdirectory>
-  </pool>
-</config>



From noreply at svn.ci.uchicago.edu  Wed Jul  9 14:48:48 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 14:48:48 -0500 (CDT)
Subject: [Swift-commit] r2103 - trunk/src/org/griphyn/vdl/karajan/lib/cache
Message-ID: <20080709194848.B8B759B4001@www.ci.uchicago.edu>

Author: hategan
Date: 2008-07-09 14:48:48 -0500 (Wed, 09 Jul 2008)
New Revision: 2103

Modified:
   trunk/src/org/griphyn/vdl/karajan/lib/cache/LRUFileCache.java
   trunk/src/org/griphyn/vdl/karajan/lib/cache/Site.java
   trunk/src/org/griphyn/vdl/karajan/lib/cache/VDLFileCache.java
Log:
added getFiles and getPaths methods

Modified: trunk/src/org/griphyn/vdl/karajan/lib/cache/LRUFileCache.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/cache/LRUFileCache.java	2008-07-09 18:18:48 UTC (rev 2102)
+++ trunk/src/org/griphyn/vdl/karajan/lib/cache/LRUFileCache.java	2008-07-09 19:48:48 UTC (rev 2103)
@@ -3,6 +3,8 @@
  */
 package org.griphyn.vdl.karajan.lib.cache;
 
+import java.util.Collection;
+import java.util.Collections;
 import java.util.HashMap;
 import java.util.Map;
 
@@ -17,22 +19,22 @@
 		Site s = getSite(entry.getHost());
 		return s.addFile(entry);
 	}
-	
+
 	public synchronized CacheReturn addAndLockEntry(File entry) {
 		Site s = getSite(entry.getHost());
 		return s.addAndLockFile(entry);
 	}
-	
+
 	public CacheReturn entryRemoved(File f) {
 		Site s = getSite(f.getHost());
 		return s.fileRemoved(f);
 	}
-	
+
 	public CacheReturn unlockEntry(File f, boolean force) {
 		Site s = getSite(f.getHost());
 		return s.unlockEntry(f, force);
 	}
-	
+
 	public CacheReturn unlockFromProcessing(File f) {
 		Site s = getSite(f.getHost());
 		return s.unlockFromProcessing(f);
@@ -48,4 +50,28 @@
 			return site;
 		}
 	}
+
+	public Collection getFiles(Object host) {
+		synchronized (sites) {
+			Site site = (Site) sites.get(host);
+			if (site == null) {
+				return Collections.EMPTY_LIST;
+			}
+			else {
+				return site.getFiles();
+			}
+		}
+	}
+	
+	public Collection getPaths(Object host) {
+		synchronized(sites) {
+			Site site = (Site) sites.get(host);
+			if (site == null) {
+				return Collections.EMPTY_LIST;
+			}
+			else {
+				return site.getPaths();
+			}
+		}
+	}
 }

Modified: trunk/src/org/griphyn/vdl/karajan/lib/cache/Site.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/cache/Site.java	2008-07-09 18:18:48 UTC (rev 2102)
+++ trunk/src/org/griphyn/vdl/karajan/lib/cache/Site.java	2008-07-09 19:48:48 UTC (rev 2103)
@@ -4,6 +4,7 @@
 package org.griphyn.vdl.karajan.lib.cache;
 
 import java.util.ArrayList;
+import java.util.Collection;
 import java.util.Collections;
 import java.util.HashMap;
 import java.util.Iterator;
@@ -157,4 +158,12 @@
 		usage = targetUsage;
 		return l;
 	}
+	
+	public synchronized Collection getFiles() {
+		return new ArrayList(files.entrySet());
+	}
+	
+	public synchronized Collection getPaths() {
+		return new ArrayList(files.keySet());
+	}
 }

Modified: trunk/src/org/griphyn/vdl/karajan/lib/cache/VDLFileCache.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/cache/VDLFileCache.java	2008-07-09 18:18:48 UTC (rev 2102)
+++ trunk/src/org/griphyn/vdl/karajan/lib/cache/VDLFileCache.java	2008-07-09 19:48:48 UTC (rev 2103)
@@ -3,6 +3,8 @@
  */
 package org.griphyn.vdl.karajan.lib.cache;
 
+import java.util.Collection;
+
 /**
  * Describes a client caching mechanism. Classes implementing this interface
  * would not deal with the actual physical data processing required to move data
@@ -11,45 +13,47 @@
  * 
  * The logical flow of the process is as follows:
  * <ol>
- * 	<li> The client code wants to cache an entry
- * 	<li> The client code calls <code>addAndLockEntry</code> with the respective
- * 		entry. This will lock the entry for both usage and processing. The
- * 		<code>addAndLockEntry</code> method returns a <code>CacheReturn</code>
- * 		object (let's call it <code>status</code>), which can signal a few things:
+ * <li> The client code wants to cache an entry
+ * <li> The client code calls <code>addAndLockEntry</code> with the respective
+ * entry. This will lock the entry for both usage and processing. The
+ * <code>addAndLockEntry</code> method returns a <code>CacheReturn</code>
+ * object (let's call it <code>status</code>), which can signal a few things:
  * 
- * 		<ol>
- * 			<li> If the entry is already cached, then <code>status.alreadyCached</code>
- * 				will be <code>true</code>, and <code>status.cached</code> will contain
- * 				the cached entry. The client code should check then if the already cached
- * 				entry is locked for processing by calling the
- * 				<code>isLockedForProcessing</code> method, and:
- * 				<ol>
- * 					<li> If the entry is locked for processing, the client code should add a
- * 						listener to the entry in order to get notified when the processing is
- * 						complete (<code>status.cached.addProcessingListener</code>). Upon receipt
- * 						of notification of processing completion, the client code should retry
- * 						calling the addAndLockEntry, since the processing may have involved the
- * 						removal of the file from the cache.
- * 					<li> If the entry is not locked for processing, the client code can assume
- * 						that the entry is correctly present in the physical cache
- * 				</ol>
- * 			<li> If the entry is not cached (<code>status.alreadyCached</code> is <code>false</code>), 
- * 				then the client code must physically put the entry into the cache, after which it 
- * 				shoult call <code>unlockFromProcessing</code>
- * 			<li> Removal requests are listed in the <code>status.remove</code> list. The entries
- * 				returned must be already locked for processing by the cache implementation, in 
- * 				order to prevent them from being used by other threads. 
- * 				The application code should physically remove such entries from the cache, and 
- * 				then call <code>fileRemoved</code>.
- * 		</ol> 
- * 	<li> The client code uses the cached data
- * 	<li> The client code calls <code>unlockEntry</code> to signal that the data is not needed by the
- * 		application any more, and it can eventually be removed from the cache
+ * <ol>
+ * <li> If the entry is already cached, then <code>status.alreadyCached</code>
+ * will be <code>true</code>, and <code>status.cached</code> will contain
+ * the cached entry. The client code should check then if the already cached
+ * entry is locked for processing by calling the
+ * <code>isLockedForProcessing</code> method, and:
+ * <ol>
+ * <li> If the entry is locked for processing, the client code should add a
+ * listener to the entry in order to get notified when the processing is
+ * complete (<code>status.cached.addProcessingListener</code>). Upon receipt
+ * of notification of processing completion, the client code should retry
+ * calling the addAndLockEntry, since the processing may have involved the
+ * removal of the file from the cache.
+ * <li> If the entry is not locked for processing, the client code can assume
+ * that the entry is correctly present in the physical cache
  * </ol>
+ * <li> If the entry is not cached (<code>status.alreadyCached</code> is
+ * <code>false</code>), then the client code must physically put the entry
+ * into the cache, after which it shoult call <code>unlockFromProcessing</code>
+ * <li> Removal requests are listed in the <code>status.remove</code> list.
+ * The entries returned must be already locked for processing by the cache
+ * implementation, in order to prevent them from being used by other threads.
+ * The application code should physically remove such entries from the cache,
+ * and then call <code>fileRemoved</code>.
+ * </ol>
+ * <li> The client code uses the cached data
+ * <li> The client code calls <code>unlockEntry</code> to signal that the data
+ * is not needed by the application any more, and it can eventually be removed
+ * from the cache
+ * </ol>
  * 
- * Alternatively, the <code>addEntry</code> method can be used to add an entry to the cache without
- * locking it in any way. This is intended for cases when the client code does not have direct control
- * over what goes into the physical cache.
+ * Alternatively, the <code>addEntry</code> method can be used to add an entry
+ * to the cache without locking it in any way. This is intended for cases when
+ * the client code does not have direct control over what goes into the physical
+ * cache.
  */
 public interface VDLFileCache {
 	/**
@@ -99,4 +103,16 @@
 	 * the entry was locked for being removed).
 	 */
 	CacheReturn unlockFromProcessing(File f);
+
+	/**
+	 * Returns a collection of {@link File} objects representing the files
+	 * stored in the cache for the specified host
+	 */
+	Collection getFiles(Object host);
+
+	/**
+	 * Returns a collection of {@link String} objects representing the file
+	 * paths stored in the cache for the specified host
+	 */
+	Collection getPaths(Object host);
 }



From noreply at svn.ci.uchicago.edu  Wed Jul  9 14:53:46 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 14:53:46 -0500 (CDT)
Subject: [Swift-commit] r2104 - SwiftApps/SIDGrid
Message-ID: <20080709195346.3271B9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-09 14:53:45 -0500 (Wed, 09 Jul 2008)
New Revision: 2104

Added:
   SwiftApps/SIDGrid/swift/
Log:
dir for project workflow development



From noreply at svn.ci.uchicago.edu  Wed Jul  9 14:56:04 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 14:56:04 -0500 (CDT)
Subject: [Swift-commit] r2105 - trunk/src/org/griphyn/vdl/karajan/lib/cache
Message-ID: <20080709195604.6C7B69B4001@www.ci.uchicago.edu>

Author: hategan
Date: 2008-07-09 14:56:04 -0500 (Wed, 09 Jul 2008)
New Revision: 2105

Modified:
   trunk/src/org/griphyn/vdl/karajan/lib/cache/VDLFileCache.java
Log:
automatic formating messes things up at times

Modified: trunk/src/org/griphyn/vdl/karajan/lib/cache/VDLFileCache.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/cache/VDLFileCache.java	2008-07-09 19:53:45 UTC (rev 2104)
+++ trunk/src/org/griphyn/vdl/karajan/lib/cache/VDLFileCache.java	2008-07-09 19:56:04 UTC (rev 2105)
@@ -13,47 +13,45 @@
  * 
  * The logical flow of the process is as follows:
  * <ol>
- * <li> The client code wants to cache an entry
- * <li> The client code calls <code>addAndLockEntry</code> with the respective
- * entry. This will lock the entry for both usage and processing. The
- * <code>addAndLockEntry</code> method returns a <code>CacheReturn</code>
- * object (let's call it <code>status</code>), which can signal a few things:
+ * 	<li> The client code wants to cache an entry
+ * 	<li> The client code calls <code>addAndLockEntry</code> with the respective
+ * 		entry. This will lock the entry for both usage and processing. The
+ * 		<code>addAndLockEntry</code> method returns a <code>CacheReturn</code>
+ * 		object (let's call it <code>status</code>), which can signal a few things:
  * 
- * <ol>
- * <li> If the entry is already cached, then <code>status.alreadyCached</code>
- * will be <code>true</code>, and <code>status.cached</code> will contain
- * the cached entry. The client code should check then if the already cached
- * entry is locked for processing by calling the
- * <code>isLockedForProcessing</code> method, and:
- * <ol>
- * <li> If the entry is locked for processing, the client code should add a
- * listener to the entry in order to get notified when the processing is
- * complete (<code>status.cached.addProcessingListener</code>). Upon receipt
- * of notification of processing completion, the client code should retry
- * calling the addAndLockEntry, since the processing may have involved the
- * removal of the file from the cache.
- * <li> If the entry is not locked for processing, the client code can assume
- * that the entry is correctly present in the physical cache
+ * 		<ol>
+ * 			<li> If the entry is already cached, then <code>status.alreadyCached</code>
+ * 				will be <code>true</code>, and <code>status.cached</code> will contain
+ * 				the cached entry. The client code should check then if the already cached
+ * 				entry is locked for processing by calling the
+ * 				<code>isLockedForProcessing</code> method, and:
+ * 				<ol>
+ * 					<li> If the entry is locked for processing, the client code should add a
+ * 						listener to the entry in order to get notified when the processing is
+ * 						complete (<code>status.cached.addProcessingListener</code>). Upon receipt
+ * 						of notification of processing completion, the client code should retry
+ * 						calling the addAndLockEntry, since the processing may have involved the
+ * 						removal of the file from the cache.
+ * 					<li> If the entry is not locked for processing, the client code can assume
+ * 						that the entry is correctly present in the physical cache
+ * 				</ol>
+ * 			<li> If the entry is not cached (<code>status.alreadyCached</code> is <code>false</code>), 
+ * 				then the client code must physically put the entry into the cache, after which it 
+ * 				shoult call <code>unlockFromProcessing</code>
+ * 			<li> Removal requests are listed in the <code>status.remove</code> list. The entries
+ * 				returned must be already locked for processing by the cache implementation, in 
+ * 				order to prevent them from being used by other threads. 
+ * 				The application code should physically remove such entries from the cache, and 
+ * 				then call <code>fileRemoved</code>.
+ * 		</ol> 
+ * 	<li> The client code uses the cached data
+ * 	<li> The client code calls <code>unlockEntry</code> to signal that the data is not needed by the
+ * 		application any more, and it can eventually be removed from the cache
  * </ol>
- * <li> If the entry is not cached (<code>status.alreadyCached</code> is
- * <code>false</code>), then the client code must physically put the entry
- * into the cache, after which it shoult call <code>unlockFromProcessing</code>
- * <li> Removal requests are listed in the <code>status.remove</code> list.
- * The entries returned must be already locked for processing by the cache
- * implementation, in order to prevent them from being used by other threads.
- * The application code should physically remove such entries from the cache,
- * and then call <code>fileRemoved</code>.
- * </ol>
- * <li> The client code uses the cached data
- * <li> The client code calls <code>unlockEntry</code> to signal that the data
- * is not needed by the application any more, and it can eventually be removed
- * from the cache
- * </ol>
  * 
- * Alternatively, the <code>addEntry</code> method can be used to add an entry
- * to the cache without locking it in any way. This is intended for cases when
- * the client code does not have direct control over what goes into the physical
- * cache.
+ * Alternatively, the <code>addEntry</code> method can be used to add an entry to the cache without
+ * locking it in any way. This is intended for cases when the client code does not have direct control
+ * over what goes into the physical cache.
  */
 public interface VDLFileCache {
 	/**
@@ -103,16 +101,16 @@
 	 * the entry was locked for being removed).
 	 */
 	CacheReturn unlockFromProcessing(File f);
-
+	
 	/**
 	 * Returns a collection of {@link File} objects representing the files
 	 * stored in the cache for the specified host
 	 */
 	Collection getFiles(Object host);
-
+	
 	/**
-	 * Returns a collection of {@link String} objects representing the file
-	 * paths stored in the cache for the specified host
+	 * Returns a collection of {@link String} objects representing the file paths
+	 * stored in the cache for the specified host
 	 */
 	Collection getPaths(Object host);
 }



From noreply at svn.ci.uchicago.edu  Wed Jul  9 14:59:29 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 14:59:29 -0500 (CDT)
Subject: [Swift-commit] r2106 - SwiftApps/SIDGrid/swift
Message-ID: <20080709195929.4D2429B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-09 14:59:28 -0500 (Wed, 09 Jul 2008)
New Revision: 2106

Added:
   SwiftApps/SIDGrid/swift/projects/
Log:
direcotry for user workflow development



From noreply at svn.ci.uchicago.edu  Wed Jul  9 15:03:29 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 15:03:29 -0500 (CDT)
Subject: [Swift-commit] r2107 - in SwiftApps/SIDGrid/swift/projects: . skenny
Message-ID: <20080709200329.0BD239B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-09 15:03:28 -0500 (Wed, 09 Jul 2008)
New Revision: 2107

Added:
   SwiftApps/SIDGrid/swift/projects/skenny/
   SwiftApps/SIDGrid/swift/projects/skenny/openmx/
Log:
skenny swift script dev dir



From noreply at svn.ci.uchicago.edu  Wed Jul  9 15:04:19 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 15:04:19 -0500 (CDT)
Subject: [Swift-commit] r2108 - SwiftApps/SIDGrid/swift/projects
Message-ID: <20080709200419.DB0499B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-09 15:04:19 -0500 (Wed, 09 Jul 2008)
New Revision: 2108

Added:
   SwiftApps/SIDGrid/swift/projects/andric/
Log:
andric swift script dev dir



From noreply at svn.ci.uchicago.edu  Wed Jul  9 15:16:57 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed,  9 Jul 2008 15:16:57 -0500 (CDT)
Subject: [Swift-commit] r2109 -
	SwiftApps/SIDGrid/swift/projects/skenny/openmx
Message-ID: <20080709201657.8DD099B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-09 15:16:57 -0500 (Wed, 09 Jul 2008)
New Revision: 2109

Added:
   SwiftApps/SIDGrid/swift/projects/skenny/openmx/modelgen.swift
Log:
generating models for openmx

Added: SwiftApps/SIDGrid/swift/projects/skenny/openmx/modelgen.swift
===================================================================
--- SwiftApps/SIDGrid/swift/projects/skenny/openmx/modelgen.swift	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/skenny/openmx/modelgen.swift	2008-07-09 20:16:57 UTC (rev 2109)
@@ -0,0 +1,26 @@
+###  generate openmx models in parallel
+###  by number of connections
+
+type file{}
+
+# produce .RData file for each model object to be tested
+
+(file mxModels) mxmodel_generator (int conn, int ncol, file dot_r, file sdat, file spdat){
+    app{
+	RInvoke @filename(dot_r) conn ncol @filename(sdat) @filename(spdat);
+    }
+}
+
+# map input and output for generator
+
+
+file modelgen<single_file_mapper;file="scripts/mxModelGen.R">;
+file sdat<single_file_mapper;file="matrices/s.dat">;
+file spdat<single_file_mapper;file="matrices/sp.dat">;
+
+	file potentialmodels<single_file_mapper;file=@strcat("models.tar")>;
+	(potentialmodels) = mxmodel_generator(4, 4, modelgen, sdat, spdat);
+
+
+
+



From noreply at svn.ci.uchicago.edu  Thu Jul 10 13:28:08 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 10 Jul 2008 13:28:08 -0500 (CDT)
Subject: [Swift-commit] r2110 -
	SwiftApps/SIDGrid/swift/projects/skenny/openmx
Message-ID: <20080710182808.5DF5F9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-10 13:28:07 -0500 (Thu, 10 Jul 2008)
New Revision: 2110

Added:
   SwiftApps/SIDGrid/swift/projects/skenny/openmx/modelproc.swift
Log:
runs RAM model for mx model on the previosly generated mx models

Added: SwiftApps/SIDGrid/swift/projects/skenny/openmx/modelproc.swift
===================================================================
--- SwiftApps/SIDGrid/swift/projects/skenny/openmx/modelproc.swift	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/skenny/openmx/modelproc.swift	2008-07-10 18:28:07 UTC (rev 2110)
@@ -0,0 +1,57 @@
+type file{}
+
+# query database for timeseries data and pass to 
+# R script to calculate observed covariance and
+# run model (RAM)
+
+(file fitmodel, file func_value) simple_covariance (string allcatargs, file config, file dot_r, file mxmodel){
+    app{
+        Mediator allcatargs @filename(dot_r) @filename(mxmodel);
+    }
+}
+
+model_data(string query, string cond, int modnum, file r_script){
+	file pmod<single_file_mapper; file=@strcat("modelObjects/",modnum,"_model.rdata")>;
+	string db = "EMBLEM1";
+	string host = "tp-neurodb.ci.uchicago.edu";
+	file config<single_file_mapper; file="user.config">;
+	string user = @arg("user");
+	string r_swift_args = @strcat(modnum,"_model.rdata");
+	string med_args = @strcat("--user ", user,
+    		" --conf ", @filename(config),
+    		" --db ", db,
+    		" --host ", host,
+    		" --query ", query,
+    		" --r_script ", @filename(r_script),
+    		" --r_swift_args ", r_swift_args,
+    		" --begin_vox ", 1,
+		" --end_vox ", 2,
+		" --batchstep ", 1,
+		" --outprefix ", @strcat(modnum,"_modeldata"),
+    		" --subject ", "model");
+	file resmod <single_file_mapper; file=@strcat(cond,"/",modnum,"/","model.rdata")>;
+	file modmin <single_file_mapper;file=@strcat(cond,"/",modnum,"/","function.min")>;
+	(resmod, modmin) = simple_covariance(med_args, config, r_script, pmod);
+}
+
+#............................main................................................
+
+# generate queries based on condition and roi's
+
+string conditions = ["gestspeech", "speech", "gesture", "grasp"];
+string queries[];
+foreach cond,c in conditions{
+	queries[c] = @strcat("SELECT avg(",cond,"0B), avg(",cond,"1B), avg(",cond,"2B), avg(",cond,"3B), avg(",cond,"4B), avg(",cond,"5B), avg(",cond,"6B), avg(",cond,"7B), avg(",cond,"8B) FROM emblemfemlh where roi = 42 or roi = 34 or roi = 33 or roi = 60 group by roi;");	
+}
+
+file cov_script<single_file_mapper; file="scripts/SimpleCovariance.R">;
+
+int models = [1:2];
+
+foreach m in models{
+	foreach query,q in queries {
+		model_data(query,conditions[q],m,cov_script);
+		}
+	    }
+
+



From noreply at svn.ci.uchicago.edu  Thu Jul 10 13:32:22 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 10 Jul 2008 13:32:22 -0500 (CDT)
Subject: [Swift-commit] r2111 - in
	SwiftApps/SIDGrid/swift/projects/skenny/openmx: . scripts
Message-ID: <20080710183222.7F0D09B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-10 13:32:22 -0500 (Thu, 10 Jul 2008)
New Revision: 2111

Added:
   SwiftApps/SIDGrid/swift/projects/skenny/openmx/scripts/
   SwiftApps/SIDGrid/swift/projects/skenny/openmx/scripts/SimpleCovariance.R
   SwiftApps/SIDGrid/swift/projects/skenny/openmx/scripts/mxModelGen.R
Log:
R scritps for processing

Added: SwiftApps/SIDGrid/swift/projects/skenny/openmx/scripts/SimpleCovariance.R
===================================================================
--- SwiftApps/SIDGrid/swift/projects/skenny/openmx/scripts/SimpleCovariance.R	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/skenny/openmx/scripts/SimpleCovariance.R	2008-07-10 18:32:22 UTC (rev 2111)
@@ -0,0 +1,34 @@
+library(OpenMx)
+
+## will need something like: R_SWIFT_ARGS=1_model.rdata
+
+inputfile <- Sys.getenv("R_INPUT")
+allinputs <- Sys.getenv("R_SWIFT_ARGS")
+
+print(allinputs)
+print(inputfile)
+
+covfile <- noquote(strsplit(inputfile," ")[[1]][1])
+modelobject <- noquote(strsplit(allinputs," ")[[1]][2])
+
+print(covfile)
+print(modelobject)
+
+covMatrix = matrix(c(read.table(covfile)))
+model = MxModel(); model = load(modelobject); model = saveLoadReference
+
+objective <- CovarianceObjective(model$cov, covMatrix)
+job <- MxJob(model, objective)
+jobClosure <- createMxClosure(job, use_R=TRUE)
+
+jobClosure();
+
+# user will get these output in the mediator's .tar file
+write.table(model$A$.values, file=paste(outname,"_asymmetric_coef.txt",sep=""), row.names=FALSE, col.names=FALSE)
+write.table(model$S$.values, file=paste(outname,"_symmetric_coef.txt",sep=""), row.names=FALSE, col.names=FALSE)
+save.Object(model,file="model.rdata")
+# swift will request this file explicitly to be used for model comparison
+# to reinvoke saved model:
+# 
+
+

Added: SwiftApps/SIDGrid/swift/projects/skenny/openmx/scripts/mxModelGen.R
===================================================================
--- SwiftApps/SIDGrid/swift/projects/skenny/openmx/scripts/mxModelGen.R	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/skenny/openmx/scripts/mxModelGen.R	2008-07-10 18:32:22 UTC (rev 2111)
@@ -0,0 +1,89 @@
+# generates all models for given number of connections
+# and writes them out as mxmodel objects
+
+library(OpenMx)
+dyn.load("/home/ac/skenny/perm.so")
+
+## should be like R_SWIFT_ARGS=3 3
+
+## get number of connections and array size (will be converted to nxn)
+
+allinputs <- Sys.getenv("R_SWIFT_ARGS")
+
+print(allinputs)
+
+connections <- noquote(strsplit(allinputs," ")[[1]][1])
+
+print(connections)
+
+numcol        <- noquote(strsplit(allinputs," ")[[1]][2])
+
+print(numcol)
+
+rowcol = as.integer(numcol)
+size = rowcol*rowcol
+
+
+# creates the .adat files 
+
+.C("run_perm", connections=as.integer(connections), size=as.integer(size))
+
+# read files and generate associated ap.adat, s.dat and sp.dat files
+# will do this in the c code
+
+# read in the data files to generate models and write as objects
+
+# dot is wildcard here, will need to adjust to it gets all 3 dat's
+count = 0
+for (i in list.files(pattern=".adat")){
+	model = MxModel()
+#	model$A <- FullMatrix(rowcol, rowcol, free=TRUE)
+#	model$S <- DiagMatrix(rowcol, rowcol) 
+	model$F <- IdenMatrix(rowcol, rowcol) # modifiable in future, for now no filter
+	model$I <- IdenMatrix(rowcol, rowcol)
+	model$cov <- MxAlgebra(model$F %&% (solve(model$I - model$A) %&% model$S))
+
+# hard-coding for now
+a = double(length = size)
+apar = double(length = size)
+xindex = array(c(1:rowcol))
+yindex = array(c(1:rowcol))
+	apar = matrix(c(read.table(i)), nrow = rowcol, ncol = rowcol)
+	cnt = rowcol+1.0
+for(x in xindex)
+{
+ for(y in yindex)
+   {
+	if (apar[x,y] == 1){
+        apar[x,y] = cnt
+	cnt = cnt + 1}
+
+   }
+}
+print(apar)
+aval = matrix(c(read.table(i)), nrow = rowcol, ncol = rowcol)
+
+for(x in xindex)
+{
+ for(y in yindex)
+   {
+	if (aval[x,y] == 1)
+        aval[x,y] = 0.75
+
+   }
+}
+print(aval)
+
+	
+	s = matrix(c(read.table("matrices/s.dat")), nrow = rowcol, ncol = rowcol)
+	spar = matrix(c(read.table("matrices/sp.dat")), nrow = rowcol, ncol = rowcol)
+	
+	model$A$values <- aval
+	model$A$parameters <- apar
+	model$S$values <- s
+	model$S$parameters <- spar
+	count = count+1
+print(model$S)
+	save.Object(model,file=paste(count,"_model.rdata", sep=""))
+}
+system("tar -cf models.tar *rdata")



From noreply at svn.ci.uchicago.edu  Thu Jul 10 15:16:40 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 10 Jul 2008 15:16:40 -0500 (CDT)
Subject: [Swift-commit] r2112 - SwiftApps/SIDGrid/uchicago_tools
Message-ID: <20080710201640.860E19B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-10 15:16:39 -0500 (Thu, 10 Jul 2008)
New Revision: 2112

Added:
   SwiftApps/SIDGrid/uchicago_tools/swift_exec.sh
Log:
simple wrapper for pointing to specific config files

Added: SwiftApps/SIDGrid/uchicago_tools/swift_exec.sh
===================================================================
--- SwiftApps/SIDGrid/uchicago_tools/swift_exec.sh	                        (rev 0)
+++ SwiftApps/SIDGrid/uchicago_tools/swift_exec.sh	2008-07-10 20:16:39 UTC (rev 2112)
@@ -0,0 +1,8 @@
+#!/bin/bash
+
+swift -tc.file /disks/gpfs/fmri/cnari/swift/config/tc.data -sites.file /disks/gpfs/fmri/cnari/swift/config/sites_ncsa.xml modelproc.swift -user="skenny"
+
+
+
+
+


Property changes on: SwiftApps/SIDGrid/uchicago_tools/swift_exec.sh
___________________________________________________________________
Name: svn:executable
   + *



From noreply at svn.ci.uchicago.edu  Thu Jul 10 16:09:33 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 10 Jul 2008 16:09:33 -0500 (CDT)
Subject: [Swift-commit] r2113 - in SwiftApps/SIDGrid/swift/projects/skenny:
	. check_env
Message-ID: <20080710210933.435299B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-10 16:09:32 -0500 (Thu, 10 Jul 2008)
New Revision: 2113

Added:
   SwiftApps/SIDGrid/swift/projects/skenny/check_env/
   SwiftApps/SIDGrid/swift/projects/skenny/check_env/env.swift
   SwiftApps/SIDGrid/swift/projects/skenny/check_env/envBatch.swift
   SwiftApps/SIDGrid/swift/projects/skenny/check_env/swift_exec.sh
   SwiftApps/SIDGrid/swift/projects/skenny/check_env/test.rsl
Log:
simple swift run for checking the environment on a worker node

Added: SwiftApps/SIDGrid/swift/projects/skenny/check_env/env.swift
===================================================================
--- SwiftApps/SIDGrid/swift/projects/skenny/check_env/env.swift	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/skenny/check_env/env.swift	2008-07-10 21:09:32 UTC (rev 2113)
@@ -0,0 +1,8 @@
+type file {}						//define a type for file
+(file t) env () { 			//procedure declaration
+    app {
+        env stdout=@filename(t);	//redirect stdout to a file
+    }
+}
+
+file e = env();		//procedure call

Added: SwiftApps/SIDGrid/swift/projects/skenny/check_env/envBatch.swift
===================================================================
--- SwiftApps/SIDGrid/swift/projects/skenny/check_env/envBatch.swift	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/skenny/check_env/envBatch.swift	2008-07-10 21:09:32 UTC (rev 2113)
@@ -0,0 +1,17 @@
+type file {}						//define a type for file
+(file t) env () { 			//procedure declaration
+    app {
+        env stdout=@filename(t);	//redirect stdout to a file
+    }
+}
+
+file e = env();
+file f = env();
+file g = env();
+file h = env();
+file i = env();
+file j = env();
+file k = env();
+file l = env();
+file m = env();
+file n = env();

Added: SwiftApps/SIDGrid/swift/projects/skenny/check_env/swift_exec.sh
===================================================================
--- SwiftApps/SIDGrid/swift/projects/skenny/check_env/swift_exec.sh	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/skenny/check_env/swift_exec.sh	2008-07-10 21:09:32 UTC (rev 2113)
@@ -0,0 +1,9 @@
+#!/bin/bash
+
+/disks/gpfs/fmri/cnari/swift/sbuilds_old/cog/modules/vdsk/dist/vdsk-svn/bin/swift -tc.file /disks/gpfs/fmri/cnari/swift/config/tc.data -sites.file ./sites_ranger.xml env.swift -user="skenny"
+
+
+
+
+
+


Property changes on: SwiftApps/SIDGrid/swift/projects/skenny/check_env/swift_exec.sh
___________________________________________________________________
Name: svn:executable
   + *

Added: SwiftApps/SIDGrid/swift/projects/skenny/check_env/test.rsl
===================================================================
--- SwiftApps/SIDGrid/swift/projects/skenny/check_env/test.rsl	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/skenny/check_env/test.rsl	2008-07-10 21:09:32 UTC (rev 2113)
@@ -0,0 +1,12 @@
+&
+
+(hostCount=4)
+
+(maxtime=5)
+
+(count=4)
+
+(project=TG-STA060008N)
+
+(executable=/bin/hostname)
+



From noreply at svn.ci.uchicago.edu  Fri Jul 11 13:04:13 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 11 Jul 2008 13:04:13 -0500 (CDT)
Subject: [Swift-commit] r2114 - SwiftApps/SIDGrid/config
Message-ID: <20080711180413.9DAEC9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-11 13:04:12 -0500 (Fri, 11 Jul 2008)
New Revision: 2114

Modified:
   SwiftApps/SIDGrid/config/sites_teraport.xml
Log:
namespace updated for gatekeeper

Modified: SwiftApps/SIDGrid/config/sites_teraport.xml
===================================================================
--- SwiftApps/SIDGrid/config/sites_teraport.xml	2008-07-10 21:09:32 UTC (rev 2113)
+++ SwiftApps/SIDGrid/config/sites_teraport.xml	2008-07-11 18:04:12 UTC (rev 2114)
@@ -9,7 +9,7 @@
     <profile namespace="env" key="wntmp">/tmp</profile>
     <profile namespace="vds" key="grid">osg</profile>
      <gridftp url="gsiftp://tp-grid1.ci.uchicago.edu" storage="/gpfs1/osg_data" major="2" minor="4" patch="3"/>
-    <jobmanager universe="vanilla" url="tp-osg.uchicago.edu/jobmanager-pbs" major="2" minor="4" patch="3"/>
+    <jobmanager universe="vanilla" url="tp-osg.ci.uchicago.edu/jobmanager-pbs" major="2" minor="4" patch="3"/>
     <workdirectory>/gpfs1/osg_data/SIDGrid/sidgrid_out</workdirectory>
   </pool>
 </config>



From noreply at svn.ci.uchicago.edu  Mon Jul 14 17:18:47 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 14 Jul 2008 17:18:47 -0500 (CDT)
Subject: [Swift-commit] r2115 -
	trunk/src/org/griphyn/vdl/karajan/lib/swiftscript
Message-ID: <20080714221847.8B2799B4001@www.ci.uchicago.edu>

Author: hategan
Date: 2008-07-14 17:18:46 -0500 (Mon, 14 Jul 2008)
New Revision: 2115

Modified:
   trunk/src/org/griphyn/vdl/karajan/lib/swiftscript/ReadData.java
Log:
improved error message when header does not match type

Modified: trunk/src/org/griphyn/vdl/karajan/lib/swiftscript/ReadData.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/swiftscript/ReadData.java	2008-07-11 18:04:12 UTC (rev 2114)
+++ trunk/src/org/griphyn/vdl/karajan/lib/swiftscript/ReadData.java	2008-07-14 22:18:46 UTC (rev 2115)
@@ -161,10 +161,14 @@
 			String[] header = line.split("\\s+");
 			Set t = new HashSet(type.getFieldNames());
 			Set h = new HashSet(Arrays.asList(header));
+			if (t.size() != h.size()) {
+				throw new ExecutionException("File header does not match type. " + "Expected "
+						+ t.size() + " whitespace separated items. Got " + h.size() + " instead.");
+			}
 			if (!t.equals(h)) {
 				throw new ExecutionException("File header does not match type. "
-						+ "Expected the following header items (in no particular order): " + t
-						+ ". Instead, the header was (again, in no particular order): " + h);
+						+ "Expected the following whitespace separated header items: " + t
+						+ ". Instead, the header was: " + h);
 			}
 			return header;
 		}



From noreply at svn.ci.uchicago.edu  Tue Jul 15 02:46:50 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 15 Jul 2008 02:46:50 -0500 (CDT)
Subject: [Swift-commit] r2116 - trunk/tests/language-behaviour
Message-ID: <20080715074651.011D09B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-15 02:46:50 -0500 (Tue, 15 Jul 2008)
New Revision: 2116

Added:
   trunk/tests/language-behaviour/076-structured-regexp-mapper.b.out.expected
   trunk/tests/language-behaviour/076-structured-regexp-mapper.swift
   trunk/tests/language-behaviour/077-regexp-mapper.foo.out.expected
   trunk/tests/language-behaviour/077-regexp-mapper.swift
Log:
bug 148. change handling of backslash symbol in string literals.

Added: trunk/tests/language-behaviour/076-structured-regexp-mapper.b.out.expected
===================================================================
--- trunk/tests/language-behaviour/076-structured-regexp-mapper.b.out.expected	                        (rev 0)
+++ trunk/tests/language-behaviour/076-structured-regexp-mapper.b.out.expected	2008-07-15 07:46:50 UTC (rev 2116)
@@ -0,0 +1 @@
+Hello, world!

Added: trunk/tests/language-behaviour/076-structured-regexp-mapper.swift
===================================================================
--- trunk/tests/language-behaviour/076-structured-regexp-mapper.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/076-structured-regexp-mapper.swift	2008-07-15 07:46:50 UTC (rev 2116)
@@ -0,0 +1,25 @@
+type messagefile {}
+
+(messagefile t) greeting() { 
+    app {
+        echo "Hello, world!" stdout=@filename(t);
+    }
+}
+
+(messagefile t) greeting2(messagefile ignored) { 
+    app {
+        echo "Hello, world!" stdout=@filename(t);
+    }
+}
+
+
+messagefile outfile <"076-structured-regexp-mapper.a.out">;
+messagefile outfile2 <structured_regexp_mapper;
+    source=outfile,
+    match="(.*)per.a(.*)",
+    transform="\\1per.b\\2"
+>;
+
+outfile = greeting();
+
+outfile2 = greeting2(outfile);

Added: trunk/tests/language-behaviour/077-regexp-mapper.foo.out.expected
===================================================================
--- trunk/tests/language-behaviour/077-regexp-mapper.foo.out.expected	                        (rev 0)
+++ trunk/tests/language-behaviour/077-regexp-mapper.foo.out.expected	2008-07-15 07:46:50 UTC (rev 2116)
@@ -0,0 +1 @@
+hello

Added: trunk/tests/language-behaviour/077-regexp-mapper.swift
===================================================================
--- trunk/tests/language-behaviour/077-regexp-mapper.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/077-regexp-mapper.swift	2008-07-15 07:46:50 UTC (rev 2116)
@@ -0,0 +1,15 @@
+type messagefile;
+
+(messagefile t) greeting() { 
+    app {
+        echo "hello" stdout=@filename(t);
+    }
+}
+
+messagefile outfile <regexp_mapper;
+       source="outfoo",
+       match="(...)(.*)",
+       transform="077-regexp-mapper.\\2.\\1">;
+
+outfile = greeting();
+



From noreply at svn.ci.uchicago.edu  Tue Jul 15 02:47:40 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 15 Jul 2008 02:47:40 -0500 (CDT)
Subject: [Swift-commit] r2117 - in trunk: resources
	tests/language/working-base
Message-ID: <20080715074740.389B49B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-15 02:47:39 -0500 (Tue, 15 Jul 2008)
New Revision: 2117

Modified:
   trunk/resources/swiftscript.g
   trunk/tests/language/working-base/039-montage.xml
   trunk/tests/language/working-base/q16.xml
   trunk/tests/language/working-base/q17.xml
Log:
part 2 of commits for bug 148. change handling of backslash symbol in string literals.

Modified: trunk/resources/swiftscript.g
===================================================================
--- trunk/resources/swiftscript.g	2008-07-15 07:46:50 UTC (rev 2116)
+++ trunk/resources/swiftscript.g	2008-07-15 07:47:39 UTC (rev 2117)
@@ -50,7 +50,9 @@
 }
 
 String quote(String s) {
-    return s.replaceAll("\\\\\"", "&quot;");
+    String s1 = s.replaceAll("\\\\\"", "&quot;");
+    String s2 = s1.replaceAll("\\\\\\\\", "\\\\");
+    return s2;
 }
 
 }

Modified: trunk/tests/language/working-base/039-montage.xml
===================================================================
--- trunk/tests/language/working-base/039-montage.xml	2008-07-15 07:46:50 UTC (rev 2116)
+++ trunk/tests/language/working-base/039-montage.xml	2008-07-15 07:47:39 UTC (rev 2117)
@@ -86,16 +86,16 @@
         <mapping descriptor="regexp_mapper">
           <param name="source"><function name="filename">
             <variableReference>img</variableReference></function></param>
-          <param name="match"><stringConstant>.*\\/(.*)</stringConstant></param>
-          <param name="transform"><stringConstant>proj_\\1</stringConstant></param>
+          <param name="match"><stringConstant>.*\/(.*)</stringConstant></param>
+          <param name="transform"><stringConstant>proj_\1</stringConstant></param>
         </mapping>
       </dataset>
       <dataset name="areaImg" type="Image">
         <mapping descriptor="regexp_mapper">
           <param name="source"><function name="filename">
             <variableReference>projImg</variableReference></function></param>
-          <param name="match"><stringConstant>(.*)\\.(.*)</stringConstant></param>
-          <param name="transform"><stringConstant>\\1_area.\\2</stringConstant></param>
+          <param name="match"><stringConstant>(.*)\.(.*)</stringConstant></param>
+          <param name="transform"><stringConstant>\1_area.\2</stringConstant></param>
         </mapping>
       </dataset>
       <call proc="mProjectPP" src="line 27">
@@ -225,8 +225,8 @@
         <mapping descriptor="regexp_mapper">
           <param name="source"><function name="filename">
             <variableReference>image1</variableReference></function></param>
-          <param name="match"><stringConstant>(.*)\\.(.*)</stringConstant></param>
-          <param name="transform"><stringConstant>\\1_area.\\2</stringConstant></param>
+          <param name="match"><stringConstant>(.*)\.(.*)</stringConstant></param>
+          <param name="transform"><stringConstant>\1_area.\2</stringConstant></param>
         </mapping>
       </dataset>
       <variable name="image2" type="Image"><structureMember>
@@ -237,8 +237,8 @@
         <mapping descriptor="regexp_mapper">
           <param name="source"><function name="filename">
             <variableReference>image2</variableReference></function></param>
-          <param name="match"><stringConstant>(.*)\\.(.*)</stringConstant></param>
-          <param name="transform"><stringConstant>\\1_area.\\2</stringConstant></param>
+          <param name="match"><stringConstant>(.*)\.(.*)</stringConstant></param>
+          <param name="transform"><stringConstant>\1_area.\2</stringConstant></param>
         </mapping>
       </dataset>
       <dataset name="diffImg" type="Image">
@@ -255,7 +255,7 @@
           <param name="source"><function name="filename">
             <variableReference>diffImg</variableReference></function></param>
           <param name="match"><stringConstant>diff(.*)fits</stringConstant></param>
-          <param name="transform"><stringConstant>fit\\1txt</stringConstant></param>
+          <param name="transform"><stringConstant>fit\1txt</stringConstant></param>
         </mapping>
       </dataset>
       <call proc="mDiffFit" src="line 59">
@@ -439,15 +439,15 @@
           <param name="source"><function name="filename">
             <variableReference>projImg</variableReference></function></param>
           <param name="match"><stringConstant>proj_(.*)</stringConstant></param>
-          <param name="transform"><stringConstant>corr_\\1</stringConstant></param>
+          <param name="transform"><stringConstant>corr_\1</stringConstant></param>
         </mapping>
       </dataset>
       <dataset name="corrArea" type="Image">
         <mapping descriptor="regexp_mapper">
           <param name="source"><function name="filename">
             <variableReference>corrImg</variableReference></function></param>
-          <param name="match"><stringConstant>(.*)\\.(.*)</stringConstant></param>
-          <param name="transform"><stringConstant>\\1_area.\\2</stringConstant></param>
+          <param name="match"><stringConstant>(.*)\.(.*)</stringConstant></param>
+          <param name="transform"><stringConstant>\1_area.\2</stringConstant></param>
         </mapping>
       </dataset>
       <call proc="mBackground" src="line 94">

Modified: trunk/tests/language/working-base/q16.xml
===================================================================
--- trunk/tests/language/working-base/q16.xml	2008-07-15 07:46:50 UTC (rev 2116)
+++ trunk/tests/language/working-base/q16.xml	2008-07-15 07:47:39 UTC (rev 2117)
@@ -38,7 +38,7 @@
       <param name="source"><function name="filename">
         <variableReference>inputfile</variableReference></function></param>
       <param name="match"><stringConstant>(.*)txt</stringConstant></param>
-      <param name="transform"><stringConstant>\\1count</stringConstant></param>
+      <param name="transform"><stringConstant>\1count</stringConstant></param>
     </mapping>
   </dataset>
   <call proc="countwords" src="line 14">

Modified: trunk/tests/language/working-base/q17.xml
===================================================================
--- trunk/tests/language/working-base/q17.xml	2008-07-15 07:46:50 UTC (rev 2116)
+++ trunk/tests/language/working-base/q17.xml	2008-07-15 07:47:39 UTC (rev 2117)
@@ -43,7 +43,7 @@
         <param name="source"><function name="filename">
           <variableReference>f</variableReference></function></param>
         <param name="match"><stringConstant>(.*)txt</stringConstant></param>
-        <param name="transform"><stringConstant>\\1count</stringConstant></param>
+        <param name="transform"><stringConstant>\1count</stringConstant></param>
       </mapping>
     </dataset>
     <call proc="countwords" src="line 17">



From noreply at svn.ci.uchicago.edu  Tue Jul 15 16:50:19 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 15 Jul 2008 16:50:19 -0500 (CDT)
Subject: [Swift-commit] r2118 - trunk/src/org/griphyn/vdl/karajan/lib/cache
Message-ID: <20080715215019.E1AB69B4001@www.ci.uchicago.edu>

Author: hategan
Date: 2008-07-15 16:50:19 -0500 (Tue, 15 Jul 2008)
New Revision: 2118

Added:
   trunk/src/org/griphyn/vdl/karajan/lib/cache/CacheMapAdapter.java
Log:
added a cache to java.util.Map adapter

Added: trunk/src/org/griphyn/vdl/karajan/lib/cache/CacheMapAdapter.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/cache/CacheMapAdapter.java	                        (rev 0)
+++ trunk/src/org/griphyn/vdl/karajan/lib/cache/CacheMapAdapter.java	2008-07-15 21:50:19 UTC (rev 2118)
@@ -0,0 +1,64 @@
+/*
+ * Created on Jul 15, 2008
+ */
+package org.griphyn.vdl.karajan.lib.cache;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+public class CacheMapAdapter implements Map {
+	private VDLFileCache cache;
+	
+	public CacheMapAdapter(VDLFileCache cache) {
+		this.cache = cache;
+	}
+
+	public void clear() {
+		throw new UnsupportedOperationException();
+	}
+
+	public boolean containsKey(Object key) {
+		throw new UnsupportedOperationException();
+	}
+
+	public boolean containsValue(Object value) {
+		throw new UnsupportedOperationException();
+	}
+
+	public Set entrySet() {
+		throw new UnsupportedOperationException();
+	}
+
+	public Object get(Object key) {
+		return cache.getPaths(key);
+	}
+
+	public boolean isEmpty() {
+		throw new UnsupportedOperationException();
+	}
+
+	public Set keySet() {
+		throw new UnsupportedOperationException();
+	}
+
+	public Object put(Object key, Object value) {
+		throw new UnsupportedOperationException();
+	}
+
+	public void putAll(Map t) {
+		throw new UnsupportedOperationException();
+	}
+
+	public Object remove(Object key) {
+		throw new UnsupportedOperationException();
+	}
+
+	public int size() {
+		throw new UnsupportedOperationException();
+	}
+
+	public Collection values() {
+		throw new UnsupportedOperationException();
+	}
+}



From noreply at svn.ci.uchicago.edu  Wed Jul 16 06:52:58 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 16 Jul 2008 06:52:58 -0500 (CDT)
Subject: [Swift-commit] r2119 - trunk/libexec
Message-ID: <20080716115258.AC02D9B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-16 06:52:58 -0500 (Wed, 16 Jul 2008)
New Revision: 2119

Modified:
   trunk/libexec/execute-default.k
   trunk/libexec/vdl-int.k
Log:
rearrange progress indicators for site selection and site retry to give a better sense of which jobs are waiting to run for the first time vs jobs which have been attempted at least once

Modified: trunk/libexec/execute-default.k
===================================================================
--- trunk/libexec/execute-default.k	2008-07-15 21:50:19 UTC (rev 2118)
+++ trunk/libexec/execute-default.k	2008-07-16 11:52:58 UTC (rev 2119)
@@ -14,6 +14,7 @@
 					sequential(
 						echo("{tr} started")
 						log(LOG:INFO, "START thread={#thread} tr={tr}") 
+						vdl:setprogress("Selecting site")
 						restartOnError(".*", vdl:configProperty("execution.retries"),
 							replicationChannel := channel()
 							discard(append(replicationChannel, true)) //trigger the first job

Modified: trunk/libexec/vdl-int.k
===================================================================
--- trunk/libexec/vdl-int.k	2008-07-15 21:50:19 UTC (rev 2118)
+++ trunk/libexec/vdl-int.k	2008-07-16 11:52:58 UTC (rev 2119)
@@ -362,7 +362,6 @@
 			replicationGroup, replicationChannel]
 			stagein := list(unique(each(stagein)))
 			stageout := list(unique(each(stageout)))
-			vdl:setprogress("Selecting site")
 			allocateHost(rhost, constraints=vdl:jobConstraints(tr)
 				
 				[wfdir, sharedDir] := try(



From noreply at svn.ci.uchicago.edu  Fri Jul 18 11:04:39 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 11:04:39 -0500 (CDT)
Subject: [Swift-commit] r2120 - SwiftApps/SIDGrid/swift/projects
Message-ID: <20080718160439.EE71F9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 11:04:39 -0500 (Fri, 18 Jul 2008)
New Revision: 2120

Added:
   SwiftApps/SIDGrid/swift/projects/uhasson/
Log:
user dir



From noreply at svn.ci.uchicago.edu  Fri Jul 18 12:40:16 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 12:40:16 -0500 (CDT)
Subject: [Swift-commit] r2122 - SwiftApps/SIDGrid/swift/projects/andric
Message-ID: <20080718174016.6D8E59B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 12:40:16 -0500 (Fri, 18 Jul 2008)
New Revision: 2122

Added:
   SwiftApps/SIDGrid/swift/projects/andric/afni_procedures/
Log:
afni workflows



From noreply at svn.ci.uchicago.edu  Fri Jul 18 12:40:53 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 12:40:53 -0500 (CDT)
Subject: [Swift-commit] r2123 - SwiftApps/SIDGrid/swift/projects/andric
Message-ID: <20080718174053.5CE1C9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 12:40:53 -0500 (Fri, 18 Jul 2008)
New Revision: 2123

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/
Log:
emblem workflows



From noreply at svn.ci.uchicago.edu  Fri Jul 18 12:41:56 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 12:41:56 -0500 (CDT)
Subject: [Swift-commit] r2124 - SwiftApps/SIDGrid/swift/projects/andric
Message-ID: <20080718174156.301BC9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 12:41:55 -0500 (Fri, 18 Jul 2008)
New Revision: 2124

Added:
   SwiftApps/SIDGrid/swift/projects/andric/db_surfing/
Log:
python scripts for database procedures



From noreply at svn.ci.uchicago.edu  Fri Jul 18 12:42:48 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 12:42:48 -0500 (CDT)
Subject: [Swift-commit] r2125 - SwiftApps/SIDGrid/swift/projects/andric
Message-ID: <20080718174248.DA3E79B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 12:42:48 -0500 (Fri, 18 Jul 2008)
New Revision: 2125

Added:
   SwiftApps/SIDGrid/swift/projects/andric/lags_repeatedmeasures/
Log:
lags



From noreply at svn.ci.uchicago.edu  Fri Jul 18 12:43:18 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 12:43:18 -0500 (CDT)
Subject: [Swift-commit] r2126 - SwiftApps/SIDGrid/swift/projects/andric
Message-ID: <20080718174318.162C89B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 12:43:17 -0500 (Fri, 18 Jul 2008)
New Revision: 2126

Added:
   SwiftApps/SIDGrid/swift/projects/andric/peakfit_HEL/
Log:
peakfit workflows



From noreply at svn.ci.uchicago.edu  Fri Jul 18 12:43:48 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 12:43:48 -0500 (CDT)
Subject: [Swift-commit] r2127 - SwiftApps/SIDGrid/swift/projects/andric
Message-ID: <20080718174348.A0A0A9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 12:43:48 -0500 (Fri, 18 Jul 2008)
New Revision: 2127

Added:
   SwiftApps/SIDGrid/swift/projects/andric/SNR/
Log:
signal to noise



From noreply at svn.ci.uchicago.edu  Fri Jul 18 12:44:45 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 12:44:45 -0500 (CDT)
Subject: [Swift-commit] r2128 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem
Message-ID: <20080718174445.6B05F9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 12:44:45 -0500 (Fri, 18 Jul 2008)
New Revision: 2128

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/
Log:
non parametric analysis



From noreply at svn.ci.uchicago.edu  Fri Jul 18 12:56:28 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 12:56:28 -0500 (CDT)
Subject: [Swift-commit] r2129 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy
Message-ID: <20080718175628.5DA8B9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 12:56:27 -0500 (Fri, 18 Jul 2008)
New Revision: 2129

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/
Log:
permutation workflows



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:01:55 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:01:55 -0500 (CDT)
Subject: [Swift-commit] r2130 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy
Message-ID: <20080718180155.E114D9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:01:55 -0500 (Fri, 18 Jul 2008)
New Revision: 2130

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/lh_NonParametric_lag_analyFILL.swift
Log:
param stuff

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/lh_NonParametric_lag_analyFILL.swift
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/lh_NonParametric_lag_analyFILL.swift	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/lh_NonParametric_lag_analyFILL.swift	2008-07-18 18:01:55 UTC (rev 2130)
@@ -0,0 +1,47 @@
+type file{}
+
+(file qout, file rout) run_query (string allcatargs, file config, file r_script){
+    app{
+        Mediator allcatargs stdout=@filename(qout) @filename(r_script);
+    }
+}
+
+string user = @arg("user");
+string db = "EMBLEM1";
+string host = "tp-neurodb.ci.uchicago.edu";
+string baseid = "lh_NonParam";
+
+file r_script<single_file_mapper; file="scripts/FriedmanTest.R">;
+file config<single_file_mapper; file="user.config">;
+
+
+loop_query(int bvox, string user, string db, string host, string query_outline, file r_script, file config, string id){
+    int evox = bvox+499;
+    string r_swift_args = @strcat(id);
+    string theoutprefix = @strcat(id,bvox,"_",evox);
+    string med_args = @strcat("--user ","andric",
+        " --conf ", "user.config",
+        " --db ", db,
+        " --host ", host,
+        " --query ", query_outline,
+        " --r_script ", @filename(r_script),
+        " --begin_vox ", bvox,
+        " --end_vox ", evox,
+        " --outprefix ", theoutprefix,
+        " --batchstep ", "500",
+        " --r_swift_args ", r_swift_args,
+        " --subject ", id);
+    file q_result <single_file_mapper; file=@strcat("results/",theoutprefix,".qresult")>;
+    file r_result <single_file_mapper; file=@strcat(theoutprefix,".txt")>;
+    (q_result, r_result) = run_query(med_args, r_script, config);
+}
+
+string regions = ["IDEAL"];
+foreach region in regions {
+    int mybatches = [4501,45501,152501];
+    foreach batch in mybatches {
+        string id = @strcat(region, baseid);
+        string query_outline = @strcat("select subject, vertex, speech_lag, emblem_lag, embspeech_lag from ccf_phase2_lh where seed_region = '",region,"' and vertex between BEGIN_BATCH and END_BATCH");
+        loop_query(batch, user, db, host, query_outline, r_script, config, id);
+    }
+}



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:02:05 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:02:05 -0500 (CDT)
Subject: [Swift-commit] r2131 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy
Message-ID: <20080718180205.C43599B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:02:05 -0500 (Fri, 18 Jul 2008)
New Revision: 2131

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/lh_NonParametric_lag_analy.swift
Log:
param stuff

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/lh_NonParametric_lag_analy.swift
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/lh_NonParametric_lag_analy.swift	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/lh_NonParametric_lag_analy.swift	2008-07-18 18:02:05 UTC (rev 2131)
@@ -0,0 +1,47 @@
+type file{}
+
+(file qout, file rout) run_query (string allcatargs, file config, file r_script){
+    app{
+        Mediator allcatargs stdout=@filename(qout) @filename(r_script);
+    }
+}
+
+string user = @arg("user");
+string db = "EMBLEM1";
+string host = "tp-neurodb.ci.uchicago.edu";
+string baseid = "lh_NonParam";
+
+file r_script<single_file_mapper; file="scripts/FriedmanTest.R">;
+file config<single_file_mapper; file="user.config">;
+
+
+loop_query(int bvox, string user, string db, string host, string query_outline, file r_script, file config, string id){
+    int evox = bvox+499;
+    string r_swift_args = @strcat(id);
+    string theoutprefix = @strcat(id,bvox,"_",evox);
+    string med_args = @strcat("--user ","andric",
+        " --conf ", "user.config",
+        " --db ", db,
+        " --host ", host,
+        " --query ", query_outline,
+        " --r_script ", @filename(r_script),
+        " --begin_vox ", bvox,
+        " --end_vox ", evox,
+        " --outprefix ", theoutprefix,
+        " --batchstep ", "500",
+        " --r_swift_args ", r_swift_args,
+        " --subject ", id);
+    file q_result <single_file_mapper; file=@strcat("results/",theoutprefix,".qresult")>;
+    file r_result <single_file_mapper; file=@strcat(theoutprefix,".txt")>;
+    (q_result, r_result) = run_query(med_args, r_script, config);
+}
+
+string regions = ["IDEAL"];
+foreach region in regions {
+    int mybatches = [1:196000:500];
+    foreach batch in mybatches {
+        string id = @strcat(region, baseid);
+        string query_outline = @strcat("select subject, vertex, speech_lag, emblem_lag, embspeech_lag from ccf_phase2_lh where seed_region = '",region,"' and vertex between BEGIN_BATCH and END_BATCH");
+        loop_query(batch, user, db, host, query_outline, r_script, config, id);
+    }
+}



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:02:27 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:02:27 -0500 (CDT)
Subject: [Swift-commit] r2132 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy
Message-ID: <20080718180227.427509B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:02:26 -0500 (Fri, 18 Jul 2008)
New Revision: 2132

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/oldsurfclust.swift
Log:
param stuff

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/oldsurfclust.swift
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/oldsurfclust.swift	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/oldsurfclust.swift	2008-07-18 18:02:26 UTC (rev 2132)
@@ -0,0 +1,34 @@
+type file{}
+
+type surfclust_obj{
+    file vals_dset;
+    file mask_dset;
+    file table;
+};
+
+
+(surfclust_obj surfclustOutput) SurfClust (file specFile, file surfFile, file input, float threshVal, string prefix){
+    app {
+        SurfClust "-spec" @filename(specFile) "-surf_A" @filename(surfFile) "-input" @filename(input) "-rmm 2.8 -thresh_col 1 -thresh" threshVal "-amm2 2 -sort_n_nodes -out_clusterdset -out_roidset -prepend_node_index -out_fulllist -prefix" prefix;
+    }
+}
+
+
+string SeedRegions = ["IDEAL"];
+string hemis = ["lh","rh"];
+float vertexThresh[] = [6.0,9.0];
+
+foreach seed in SeedRegions{
+    foreach h in hemis{
+        foreach thresh in vertexThresh{
+            string HORRYsurf_dir = @strcat("/disks/gpfs/fmri/NL-EMB/fs.subjs/HORRY/SUMA/");
+            file HORRYspec <simple_mapper;location=HORRYsurf_dir, file=@strcat("HORRY.",h,".mesh140_std.spec")>;
+            file HORRYsurf <simple_mapper;location=HORRYsurf_dir, file=@strcat("HORRY.",h,".mesh140_std.pial.asc")>;
+            string inputName=@strcat("results/",h,"_",seed,"_new_sorted.txt");
+            file inputFile <single_file_mapper;file=inputName>;
+            string outPrefix=@strcat(h,"_",seed,"_thresh",thresh);
+            surfclust_obj surfclustOutput <simple_mapper;prefix=outPrefix>;
+            (surfclustOutput) = SurfClust(HORRYspec,HORRYsurf,inputFile,thresh,outPrefix);
+        }
+    }
+}



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:02:42 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:02:42 -0500 (CDT)
Subject: [Swift-commit] r2133 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy
Message-ID: <20080718180242.C75189B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:02:42 -0500 (Fri, 18 Jul 2008)
New Revision: 2133

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permclust.swift
Log:
param stuff

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permclust.swift
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permclust.swift	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permclust.swift	2008-07-18 18:02:42 UTC (rev 2133)
@@ -0,0 +1,29 @@
+type file{}
+
+type surfclust_table{}
+
+
+(surfclust_table surfclustOutput) SurfClust (file specFile, file surfFile, file input, float threshVal, string prefix){
+    app {
+        SurfClust "-spec" @filename(specFile) "-surf_A" @filename(surfFile) "-input" @filename(input) "1 -rmm 2.8 -thresh_col 1 -thresh" threshVal "-amm2 2 -sort_n_nodes -prepend_node_index -prefix" prefix;
+    }
+}
+
+
+string SeedRegions = ["IDEAL"];
+string hemis = ["lh","rh"];
+float vertexThresh[] = [6.0];
+
+foreach seed in SeedRegions{
+    foreach h in hemis{
+        foreach thresh in vertexThresh{
+            file HORRYspec <single_file_mapper;file=@strcat("HORRY.",h,".mesh140_std.spec")>;
+            file HORRYsurf <single_file_mapper;file=@strcat("HORRY.",h,".mesh140_std.pial.asc")>;
+            string inputName=@strcat("results/",h,"_",seed,"_new_sorted.txt");
+            file inputFile <single_file_mapper;file=inputName>;
+            string outPrefix=@strcat(h,"_",seed,"_PERMTEST",thresh);
+            surfclust_table surfclustOutput <single_file_mapper; file=@strcat(h,"_",seed,"_PERMTEST6_ClstTable_r2.8_a2.0.1D")>;
+            (surfclustOutput) = SurfClust(HORRYspec,HORRYsurf,inputFile,thresh,outPrefix);
+        }
+    }
+}



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:02:56 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:02:56 -0500 (CDT)
Subject: [Swift-commit] r2134 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy
Message-ID: <20080718180256.4CAC89B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:02:55 -0500 (Fri, 18 Jul 2008)
New Revision: 2134

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/rh_NonParametric_lag_analy.swift
Log:
param stuff

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/rh_NonParametric_lag_analy.swift
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/rh_NonParametric_lag_analy.swift	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/rh_NonParametric_lag_analy.swift	2008-07-18 18:02:55 UTC (rev 2134)
@@ -0,0 +1,47 @@
+type file{}
+
+(file qout, file rout) run_query (string allcatargs, file config, file r_script){
+    app{
+        Mediator allcatargs stdout=@filename(qout) @filename(r_script);
+    }
+}
+
+string user = @arg("user");
+string db = "EMBLEM1";
+string host = "tp-neurodb.ci.uchicago.edu";
+string baseid = "rh_NonParam";
+
+file r_script<single_file_mapper; file="scripts/FriedmanTest.R">;
+file config<single_file_mapper; file="user.config">;
+
+
+loop_query(int bvox, string user, string db, string host, string query_outline, file r_script, file config, string id){
+    int evox = bvox+499;
+    string r_swift_args = @strcat(id);
+    string theoutprefix = @strcat(id,bvox,"_",evox);
+    string med_args = @strcat("--user ","andric",
+        " --conf ", "user.config",
+        " --db ", db,
+        " --host ", host,
+        " --query ", query_outline,
+        " --r_script ", @filename(r_script),
+        " --begin_vox ", bvox,
+        " --end_vox ", evox,
+        " --outprefix ", theoutprefix,
+        " --batchstep ", "500",
+        " --r_swift_args ", r_swift_args,
+        " --subject ", id);
+    file q_result <single_file_mapper; file=@strcat("results/",theoutprefix,".qresult")>;
+    file r_result <single_file_mapper; file=@strcat(theoutprefix,".txt")>;
+    (q_result, r_result) = run_query(med_args, r_script, config);
+}
+
+string regions = ["IDEAL"];
+foreach region in regions {
+    int mybatches = [1:196000:500];
+    foreach batch in mybatches {
+        string id = @strcat(region, baseid);
+        string query_outline = @strcat("select subject, vertex, speech_lag, emblem_lag, embspeech_lag from ccf_phase2_rh where seed_region = '",region,"' and vertex between BEGIN_BATCH and END_BATCH");
+        loop_query(batch, user, db, host, query_outline, r_script, config, id);
+    }
+}



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:08:35 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:08:35 -0500 (CDT)
Subject: [Swift-commit] r2135 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations
Message-ID: <20080718180835.17C579B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:08:34 -0500 (Fri, 18 Jul 2008)
New Revision: 2135

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/
Log:
input scripts to swift



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:17:36 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:17:36 -0500 (CDT)
Subject: [Swift-commit] r2136 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts
Message-ID: <20080718181736.5462B9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:17:35 -0500 (Fri, 18 Jul 2008)
New Revision: 2136

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/cat_sort.py
Log:
sorts something or other

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/cat_sort.py
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/cat_sort.py	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/cat_sort.py	2008-07-18 18:17:35 UTC (rev 2136)
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+
+import os
+import sys
+import commands
+
+
+brains = range(3,3001)
+
+os.chdir(os.getenv("cnari")+"/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/")
+print os.getcwd()
+
+for id in brains:
+    print "brain: "+`id`
+    commands.getoutput("cat "+`id`+"PermFriedman* | sort -g -k 1 > brain_"+`id`+"PermFriedman.txt")
+    commands.getoutput("rm -rf "+`id`+"PermFriedman*")



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:17:54 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:17:54 -0500 (CDT)
Subject: [Swift-commit] r2137 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts
Message-ID: <20080718181754.28A1D9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:17:53 -0500 (Fri, 18 Jul 2008)
New Revision: 2137

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/generateConfigFile.R
Log:
config generator

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/generateConfigFile.R
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/generateConfigFile.R	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/generateConfigFile.R	2008-07-18 18:17:53 UTC (rev 2137)
@@ -0,0 +1,9 @@
+cc <- as.vector(c(rep(0,12),rep(1,12)))
+con <- matrix(nrow=4000,ncol=27)
+mat_row <- 0;
+
+for (i in 1:nrow(con)){
+    mat_row <- mat_row + 1
+    con[mat_row,] <- c(sample(cc),sample(1:3))
+}
+write.table(con,file="perm_config_file.txt",row.names=FALSE,col.names=FALSE,quote=FALSE)



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:18:20 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:18:20 -0500 (CDT)
Subject: [Swift-commit] r2138 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts
Message-ID: <20080718181820.F0A669B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:18:20 -0500 (Fri, 18 Jul 2008)
New Revision: 2138

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/oldpermFriedman.R
Log:
perm stuff

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/oldpermFriedman.R
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/oldpermFriedman.R	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/oldpermFriedman.R	2008-07-18 18:18:20 UTC (rev 2138)
@@ -0,0 +1,31 @@
+#---- this is to do friedman non-parametric permutation tests
+allinputs <- Sys.getenv("R_SWIFT_ARGS")
+print(Sys.getenv("R_SWIFT_ARGS"));
+brain <- noquote(strsplit(allinputs," ")[[1]][1])
+perm_start <- as.integer(noquote(strsplit(allinputs," ")[[1]][2]))
+print(brain)
+print(perm_start)
+print(date())
+outname <- paste("outputs/",brain,"_batchstart",perm_start,"output.txt",sep="");
+print(outname)
+
+m <- matrix(nrow=7000,ncol=2);
+perm_end <- perm_start + (nrow(m)-1);
+mat_row <- 0;
+for (vert in perm_start:perm_end){
+    perm_mat <- matrix(nrow=24,ncol=3)
+    perm_mat_col = 1
+    for (col in 1:ncol(perm_mat)){
+        for (i in 1:nrow(perm_mat)){
+            perm_mat[i,col] <- sample(c(-2:2),1)
+        }
+    }
+    mat_row = mat_row + 1
+    friedman_test <- friedman.test(perm_mat)
+    vert_id <- as.integer(vert)
+    m[mat_row, ] <- c(vert_id, friedman_test$statistic[[1]])
+    rm(perm_mat)
+}
+m <- round(m,5)
+print(date())
+write.table(m, file=outname, row.names=FALSE, col.names=FALSE, quote=F)



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:18:45 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:18:45 -0500 (CDT)
Subject: [Swift-commit] r2139 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts
Message-ID: <20080718181845.7F64A9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:18:45 -0500 (Fri, 18 Jul 2008)
New Revision: 2139

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/permFriedmanDRAFT1.R
Log:
perm stuff

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/permFriedmanDRAFT1.R
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/permFriedmanDRAFT1.R	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/permFriedmanDRAFT1.R	2008-07-18 18:18:45 UTC (rev 2139)
@@ -0,0 +1,34 @@
+#---- doing friedman permutations
+#---- coded Wednesday; July 2, 2008
+
+#---- this is the function used in the aggregate:
+FriedmanPerm <- function(x){
+    mm <- matrix(nrow=24,ncol=3)
+    mm[,1] = x[1:24]
+    mm[,2] = x[25:48]
+    mm[,3] = x[49:72]
+    a <- mm[1:12,sample(1:3)]
+    b <- mm[13:24,]
+    mm_shuffle <- rbind(a,b)
+    return(friedman.test(mm_shuffle)[[1]][[1]])
+}
+
+#---- Swift housekeeping:
+allinputs <- Sys.getenv("R_SWIFT_ARGS")
+print(Sys.getenv("R_SWIFT_ARGS"));
+outname <- noquote(strsplit(allinputs," ")[[1]][1])
+outname2 <- noquote(strsplit(allinputs," ")[[1]][2])
+
+inputfile <- Sys.getenv("R_INPUT")
+print(inputfile)
+Query_out <- as.matrix(read.table(inputfile))
+
+
+#---- this is where the fun begins:
+data_stack <- stack(data.frame(Query_out[,3:5]))[1]
+factors <- Query_out[,2]
+nn <- data.frame(cbind(factors,data_stack))
+m <- matrix(nrow=length(as.integer(levels(as.factor(factors)))),ncol=2)
+m[,1] <- as.integer(levels(as.factor(factors)))
+m[,2] <- aggregate(nn$values, list(nn$factors),FriedmanPerm)[,2]
+write.table(round(m,5), file=paste(outname2,outname,".txt",sep=""), row.names=FALSE, col.names=FALSE, quote=F)



From noreply at svn.ci.uchicago.edu  Fri Jul 18 13:19:03 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Fri, 18 Jul 2008 13:19:03 -0500 (CDT)
Subject: [Swift-commit] r2140 -
	SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts
Message-ID: <20080718181903.9140F9B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-18 13:19:03 -0500 (Fri, 18 Jul 2008)
New Revision: 2140

Added:
   SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/permFriedman.R
Log:
perm stuff

Added: SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/permFriedman.R
===================================================================
--- SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/permFriedman.R	                        (rev 0)
+++ SwiftApps/SIDGrid/swift/projects/andric/ccf_emblem/NonParametric_lag_analy/permutations/scripts/permFriedman.R	2008-07-18 18:19:03 UTC (rev 2140)
@@ -0,0 +1,39 @@
+#---- doing friedman permutations
+#---- coded Wednesday; July 2, 2008
+
+#---- this is the function used in the aggregate:
+FriedmanPerm <- function(x){
+    mm <- matrix(nrow=24,ncol=3)
+    mm[,1] = x[1:24]
+    mm[,2] = x[25:48]
+    mm[,3] = x[49:72]
+    a <- mm[subj_resamp,cond_resamp]
+    b <- mm[subj_same,]
+    mm_shuffle <- rbind(a,b)
+    return(friedman.test(mm_shuffle)[[1]][[1]])
+}
+
+#---- Swift housekeeping:
+allinputs <- Sys.getenv("R_SWIFT_ARGS")
+print(Sys.getenv("R_SWIFT_ARGS"));
+outname <- noquote(strsplit(allinputs," ")[[1]][1])
+outname2 <- noquote(strsplit(allinputs," ")[[1]][2])
+configLine <- noquote(strsplit(allinputs," ")[[1]][3])
+
+inputfile <- Sys.getenv("R_INPUT")
+print(inputfile)
+Query_out <- as.matrix(read.table(inputfile))
+config <- as.matrix(read.table("perm_config_file.txt"))
+cond_resamp <- as.vector(config[configLine,25:27])
+subj_resamp <- which(as.vector(config[configLine,1:24])==1)
+subj_same <- which(as.vector(config[configLine,1:24])==0)
+
+
+#---- this is where the fun begins:
+data_stack <- stack(data.frame(Query_out[,3:5]))[1]
+vertices <- Query_out[,2]
+nn <- data.frame(cbind(vertices,data_stack))
+m <- matrix(nrow=length(as.integer(levels(as.factor(vertices)))),ncol=2)
+m[,1] <- as.integer(levels(as.factor(vertices)))
+m[,2] <- aggregate(nn$values, list(nn$vertices),FriedmanPerm)[,2]
+write.table(round(m,5), file=paste(outname2,outname,".txt",sep=""), row.names=FALSE, col.names=FALSE, quote=F)



From noreply at svn.ci.uchicago.edu  Mon Jul 21 14:23:25 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 21 Jul 2008 14:23:25 -0500 (CDT)
Subject: [Swift-commit] r2141 - trunk
Message-ID: <20080721192325.AD5B09B4001@www.ci.uchicago.edu>

Author: hategan
Date: 2008-07-21 14:23:24 -0500 (Mon, 21 Jul 2008)
New Revision: 2141

Modified:
   trunk/CHANGES.txt
Log:
testing commit

Modified: trunk/CHANGES.txt
===================================================================
--- trunk/CHANGES.txt	2008-07-18 18:19:03 UTC (rev 2140)
+++ trunk/CHANGES.txt	2008-07-21 19:23:24 UTC (rev 2141)
@@ -1,3 +1,4 @@
+
 (06/06/08)
 *** SWIFT_JOBDIR_PATH environment variable can be use in profiles (or
     anywhere else that will get it to the worker script) to cause the



From noreply at svn.ci.uchicago.edu  Mon Jul 21 17:42:44 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 21 Jul 2008 17:42:44 -0500 (CDT)
Subject: [Swift-commit] r2142 - trunk
Message-ID: <20080721224244.C1A5A9B4001@www.ci.uchicago.edu>

Author: hategan
Date: 2008-07-21 17:42:44 -0500 (Mon, 21 Jul 2008)
New Revision: 2142

Modified:
   trunk/CHANGES.txt
Log:
testing commits II

Modified: trunk/CHANGES.txt
===================================================================
--- trunk/CHANGES.txt	2008-07-21 19:23:24 UTC (rev 2141)
+++ trunk/CHANGES.txt	2008-07-21 22:42:44 UTC (rev 2142)
@@ -1,4 +1,3 @@
-
 (06/06/08)
 *** SWIFT_JOBDIR_PATH environment variable can be use in profiles (or
     anywhere else that will get it to the worker script) to cause the



From noreply at svn.ci.uchicago.edu  Mon Jul 21 20:11:29 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 21 Jul 2008 20:11:29 -0500 (CDT)
Subject: [Swift-commit] r2143 - SwiftApps/SIDGrid/config
Message-ID: <20080722011129.942889B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-21 20:11:29 -0500 (Mon, 21 Jul 2008)
New Revision: 2143

Modified:
   SwiftApps/SIDGrid/config/tc.data
Log:
ucanl64 re-added

Modified: SwiftApps/SIDGrid/config/tc.data
===================================================================
--- SwiftApps/SIDGrid/config/tc.data	2008-07-21 22:42:44 UTC (rev 2142)
+++ SwiftApps/SIDGrid/config/tc.data	2008-07-22 01:11:29 UTC (rev 2143)
@@ -1715,3 +1715,10 @@
 RANGER env /usr/bin/env  INSTALLED INTEL64::LINUX   null
 RANGER Mediator /scratch/projects/tg/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX   env::PATHPREFIX=/scratch/projects/tg/SIDGrid/usr/bin;GLOBUS::queue=development
 
+####-----------------ANLUCTERAGRID64-----------------------------
+
+ANLUCTERAGRID64   Mediator /soft/community/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX  null
+ANLUCTERAGRID64   echo /bin/echo  INSTALLED INTEL64::LINUX  null
+ANLUCTERAGRID64   RInvoke /soft/community/SIDGrid/scripts/RInvoke.sh  INSTALLED INTEL64::LINUX  null
+ANLUCTERAGRID64   env /bin/env INSTALLED INTEL64::LINUX  null
+



From noreply at svn.ci.uchicago.edu  Mon Jul 21 20:14:59 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 21 Jul 2008 20:14:59 -0500 (CDT)
Subject: [Swift-commit] r2144 - SwiftApps/SIDGrid/config
Message-ID: <20080722011459.205B39B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-21 20:14:58 -0500 (Mon, 21 Jul 2008)
New Revision: 2144

Modified:
   SwiftApps/SIDGrid/config/tc.data
Log:
tungsten retired

Modified: SwiftApps/SIDGrid/config/tc.data
===================================================================
--- SwiftApps/SIDGrid/config/tc.data	2008-07-22 01:11:29 UTC (rev 2143)
+++ SwiftApps/SIDGrid/config/tc.data	2008-07-22 01:14:58 UTC (rev 2144)
@@ -866,284 +866,6 @@
 UCTERAPORT	RInvoke		/gpfs1/osg_data/SIDGrid/scripts/RInvoke.sh   INSTALLED INTEL64::LINUX GLOBUS::maxwalltime=300,queue=short
 UCTERAPORT	Mediator	/gpfs1/osg_data/SIDGrid/scripts/Mediator.py INSTALLED INTEL64::LINUX env::PATHPREFIX=/gpfs1/osg_data/SIDGrid/usr/bin;GLOBUS::maxwalltime=300,queue=short
 
-####-----------------NCSATUNG-----------------------------
-NCSATUNG   AFNI_1dcat   /usr/projects/sidgrid/usr/bin/1dcat   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1ddot   /usr/projects/sidgrid/usr/bin/1ddot   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1deval   /usr/projects/sidgrid/usr/bin/1deval   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1dfft   /usr/projects/sidgrid/usr/bin/1dfft   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1dgrayplot   /usr/projects/sidgrid/usr/bin/1dgrayplot   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1dmatcalc   /usr/projects/sidgrid/usr/bin/1dmatcalc   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1dnorm   /usr/projects/sidgrid/usr/bin/1dnorm   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1dplot   /usr/projects/sidgrid/usr/bin/1dplot   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1dsum   /usr/projects/sidgrid/usr/bin/1dsum   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1dsvd   /usr/projects/sidgrid/usr/bin/1dsvd   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_1dtranspose   /usr/projects/sidgrid/usr/bin/1dtranspose   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_24swap   /usr/projects/sidgrid/usr/bin/24swap   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_2dImReg   /usr/projects/sidgrid/usr/bin/2dImReg   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_2dwarper   /usr/projects/sidgrid/usr/bin/@2dwarper   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_2swap   /usr/projects/sidgrid/usr/bin/2swap   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAcost   /usr/projects/sidgrid/usr/bin/3dAcost   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAFNIto3D   /usr/projects/sidgrid/usr/bin/3dAFNIto3D   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAFNItoANALYZE   /usr/projects/sidgrid/usr/bin/3dAFNItoANALYZE   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAFNItoMINC   /usr/projects/sidgrid/usr/bin/3dAFNItoMINC   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAFNItoNIFTI   /usr/projects/sidgrid/usr/bin/3dAFNItoNIFTI   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAFNItoNIML   /usr/projects/sidgrid/usr/bin/3dAFNItoNIML   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAFNItoRaw   /usr/projects/sidgrid/usr/bin/3dAFNItoRaw   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAllineate   /usr/projects/sidgrid/usr/bin/3dAllineate   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dANALYZEtoAFNI   /usr/projects/sidgrid/usr/bin/3dANALYZEtoAFNI   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAnatNudge   /usr/projects/sidgrid/usr/bin/3dAnatNudge   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAnhist   /usr/projects/sidgrid/usr/bin/3dAnhist   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3danisosmooth   /usr/projects/sidgrid/usr/bin/3danisosmooth   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dANOVA   /usr/projects/sidgrid/usr/bin/3dANOVA   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dANOVA2   /usr/projects/sidgrid/usr/bin/3dANOVA2   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dANOVA3   /usr/projects/sidgrid/usr/bin/3dANOVA3   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAttribute   /usr/projects/sidgrid/usr/bin/3dAttribute   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAutobox   /usr/projects/sidgrid/usr/bin/3dAutobox   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAutomask   /usr/projects/sidgrid/usr/bin/3dAutomask   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dAutoTcorrelate   /usr/projects/sidgrid/usr/bin/3dAutoTcorrelate   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3daxialize   /usr/projects/sidgrid/usr/bin/3daxialize   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dBlurToFWHM   /usr/projects/sidgrid/usr/bin/3dBlurToFWHM   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dBRAIN_VOYAGERtoAFNI   /usr/projects/sidgrid/usr/bin/3dBRAIN_VOYAGERtoAFNI   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dBrickStat   /usr/projects/sidgrid/usr/bin/3dBrickStat   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dbuc2fim   /usr/projects/sidgrid/usr/bin/3dbuc2fim   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dbucket   /usr/projects/sidgrid/usr/bin/3dbucket   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dcalc   /usr/projects/sidgrid/usr/bin/3dcalc   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dClipLevel   /usr/projects/sidgrid/usr/bin/3dClipLevel   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dclust   /usr/projects/sidgrid/usr/bin/3dclust   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dCM   /usr/projects/sidgrid/usr/bin/3dCM   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dConvolve   /usr/projects/sidgrid/usr/bin/3dConvolve   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dcopy   /usr/projects/sidgrid/usr/bin/3dcopy   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dCRUISEtoAFNI   /usr/projects/sidgrid/usr/bin/3dCRUISEtoAFNI   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dDeconvolve   /usr/projects/sidgrid/usr/bin/3dDeconvolve   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dDeconvolve_f   /usr/projects/sidgrid/usr/bin/3dDeconvolve_f   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3ddelay   /usr/projects/sidgrid/usr/bin/3ddelay   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dDespike   /usr/projects/sidgrid/usr/bin/3dDespike   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dDetrend   /usr/projects/sidgrid/usr/bin/3dDetrend   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3ddot   /usr/projects/sidgrid/usr/bin/3ddot   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dDTeig   /usr/projects/sidgrid/usr/bin/3dDTeig   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3ddup   /usr/projects/sidgrid/usr/bin/3ddup   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dDWItoDT   /usr/projects/sidgrid/usr/bin/3dDWItoDT   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dedge3   /usr/projects/sidgrid/usr/bin/3dedge3   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dEntropy   /usr/projects/sidgrid/usr/bin/3dEntropy   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dExtrema   /usr/projects/sidgrid/usr/bin/3dExtrema   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dFDR   /usr/projects/sidgrid/usr/bin/3dFDR   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dfimplus   /usr/projects/sidgrid/usr/bin/3dfim+   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dfim   /usr/projects/sidgrid/usr/bin/3dfim   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dFourier   /usr/projects/sidgrid/usr/bin/3dFourier   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dfractionize   /usr/projects/sidgrid/usr/bin/3dfractionize   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dFriedman   /usr/projects/sidgrid/usr/bin/3dFriedman   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dFWHM   /usr/projects/sidgrid/usr/bin/3dFWHM   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dFWHMx   /usr/projects/sidgrid/usr/bin/3dFWHMx   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dGetrow   /usr/projects/sidgrid/usr/bin/3dGetrow   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dhistog   /usr/projects/sidgrid/usr/bin/3dhistog   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dinfo   /usr/projects/sidgrid/usr/bin/3dinfo   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dIntracranial   /usr/projects/sidgrid/usr/bin/3dIntracranial   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dInvFMRI   /usr/projects/sidgrid/usr/bin/3dInvFMRI   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dKruskalWallis   /usr/projects/sidgrid/usr/bin/3dKruskalWallis   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dLocalBistat   /usr/projects/sidgrid/usr/bin/3dLocalBistat   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dLocalstat   /usr/projects/sidgrid/usr/bin/3dLocalstat   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dLRflip   /usr/projects/sidgrid/usr/bin/3dLRflip   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dMannWhitney   /usr/projects/sidgrid/usr/bin/3dMannWhitney   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dmaskave   /usr/projects/sidgrid/usr/bin/3dmaskave   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dmaskdump   /usr/projects/sidgrid/usr/bin/3dmaskdump   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dmatcalc   /usr/projects/sidgrid/usr/bin/3dmatcalc   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dMax   /usr/projects/sidgrid/usr/bin/3dMax   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dmaxima   /usr/projects/sidgrid/usr/bin/3dmaxima   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dMean   /usr/projects/sidgrid/usr/bin/3dMean   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dMedianFilter   /usr/projects/sidgrid/usr/bin/3dMedianFilter   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dmerge   /usr/projects/sidgrid/usr/bin/3dmerge   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dMINCtoAFNI   /usr/projects/sidgrid/usr/bin/3dMINCtoAFNI   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dnewid   /usr/projects/sidgrid/usr/bin/3dnewid   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dNLfim   /usr/projects/sidgrid/usr/bin/3dNLfim   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dnoise   /usr/projects/sidgrid/usr/bin/3dnoise   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dNotes   /usr/projects/sidgrid/usr/bin/3dNotes   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dnvals   /usr/projects/sidgrid/usr/bin/3dnvals   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dOverlap   /usr/projects/sidgrid/usr/bin/3dOverlap   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dpc   /usr/projects/sidgrid/usr/bin/3dpc   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dproject   /usr/projects/sidgrid/usr/bin/3dproject   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3drefit   /usr/projects/sidgrid/usr/bin/3drefit   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dRegAna   /usr/projects/sidgrid/usr/bin/3dRegAna   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3drename   /usr/projects/sidgrid/usr/bin/3drename   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dresample   /usr/projects/sidgrid/usr/bin/3dresample   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dretroicor   /usr/projects/sidgrid/usr/bin/3dretroicor   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dROIstats   /usr/projects/sidgrid/usr/bin/3dROIstats   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3drotate   /usr/projects/sidgrid/usr/bin/3drotate   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dRowFillin   /usr/projects/sidgrid/usr/bin/3dRowFillin   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dSkullStrip   /usr/projects/sidgrid/usr/bin/3dSkullStrip   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dSpatNorm   /usr/projects/sidgrid/usr/bin/3dSpatNorm   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dStatClust   /usr/projects/sidgrid/usr/bin/3dStatClust   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dSurf2Vol   /usr/projects/sidgrid/usr/bin/3dSurf2Vol   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dSurfMask   /usr/projects/sidgrid/usr/bin/3dSurfMask   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dTagalign   /usr/projects/sidgrid/usr/bin/3dTagalign   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dTcat   /usr/projects/sidgrid/usr/bin/3dTcat   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dTcorrelate   /usr/projects/sidgrid/usr/bin/3dTcorrelate   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dThreetoRGB   /usr/projects/sidgrid/usr/bin/3dThreetoRGB   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dToutcount   /usr/projects/sidgrid/usr/bin/3dToutcount   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dTqual   /usr/projects/sidgrid/usr/bin/3dTqual   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dTSgen   /usr/projects/sidgrid/usr/bin/3dTSgen   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dTshift   /usr/projects/sidgrid/usr/bin/3dTshift   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dTsmooth   /usr/projects/sidgrid/usr/bin/3dTsmooth   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dTstat   /usr/projects/sidgrid/usr/bin/3dTstat   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dttest   /usr/projects/sidgrid/usr/bin/3dttest   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dTwotoComplex   /usr/projects/sidgrid/usr/bin/3dTwotoComplex   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dUndump   /usr/projects/sidgrid/usr/bin/3dUndump   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dUniformize   /usr/projects/sidgrid/usr/bin/3dUniformize   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dVol2Surf   /usr/projects/sidgrid/usr/bin/3dVol2Surf   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dvolreg   /usr/projects/sidgrid/usr/bin/3dvolreg   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dWarp   /usr/projects/sidgrid/usr/bin/3dWarp   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dWarpDrive   /usr/projects/sidgrid/usr/bin/3dWarpDrive   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dWavelets   /usr/projects/sidgrid/usr/bin/3dWavelets   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dWilcoxon   /usr/projects/sidgrid/usr/bin/3dWilcoxon   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dWinsor   /usr/projects/sidgrid/usr/bin/3dWinsor   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dZcat   /usr/projects/sidgrid/usr/bin/3dZcat   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dZcutup   /usr/projects/sidgrid/usr/bin/3dZcutup   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dZeropad   /usr/projects/sidgrid/usr/bin/3dZeropad   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_3dZregrid   /usr/projects/sidgrid/usr/bin/3dZregrid   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_4Daverage   /usr/projects/sidgrid/usr/bin/@4Daverage   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_4swap   /usr/projects/sidgrid/usr/bin/4swap   INSTALLED INTEL32::LINUX null
-NCSATUNG   abut   /usr/projects/sidgrid/usr/bin/abut   INSTALLED INTEL32::LINUX null
-NCSATUNG   adwarp   /usr/projects/sidgrid/usr/bin/adwarp   INSTALLED INTEL32::LINUX null
-NCSATUNG   afni   /usr/projects/sidgrid/usr/bin/afni   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_afni_base_py   /usr/projects/sidgrid/usr/bin/afni_base.py   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_AfniOrient2RAImap   /usr/projects/sidgrid/usr/bin/@AfniOrient2RAImap   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_AfniOrientSign   /usr/projects/sidgrid/usr/bin/@AfniOrientSign   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_afni_proc_py   /usr/projects/sidgrid/usr/bin/afni_proc.py   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_afni_util_py   /usr/projects/sidgrid/usr/bin/afni_util.py   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_afni_vcheck   /usr/projects/sidgrid/usr/bin/afni_vcheck   INSTALLED INTEL32::LINUX null
-NCSATUNG   aiv   /usr/projects/sidgrid/usr/bin/aiv   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_Align_Centers   /usr/projects/sidgrid/usr/bin/@Align_Centers   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_align_partial_oblique   /usr/projects/sidgrid/usr/bin/@align_partial_oblique   INSTALLED INTEL32::LINUX null
-NCSATUNG   AlphaSim   /usr/projects/sidgrid/usr/bin/AlphaSim   INSTALLED INTEL32::LINUX null
-NCSATUNG   AnalyzeTrace   /usr/projects/sidgrid/usr/bin/AnalyzeTrace   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_ask_me_py   /usr/projects/sidgrid/usr/bin/ask_me.py   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_auto_align   /usr/projects/sidgrid/usr/bin/@auto_align   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_auto_tlrc   /usr/projects/sidgrid/usr/bin/@auto_tlrc   INSTALLED INTEL32::LINUX null
-NCSATUNG   byteorder   /usr/projects/sidgrid/usr/bin/byteorder   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_cat_matvec   /usr/projects/sidgrid/usr/bin/cat_matvec   INSTALLED INTEL32::LINUX null
-NCSATUNG   ccalc   /usr/projects/sidgrid/usr/bin/ccalc   INSTALLED INTEL32::LINUX null
-NCSATUNG   cdf   /usr/projects/sidgrid/usr/bin/cdf   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_Center_Distance   /usr/projects/sidgrid/usr/bin/@Center_Distance   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_CheckForAfniDset   /usr/projects/sidgrid/usr/bin/@CheckForAfniDset   INSTALLED INTEL32::LINUX null
-NCSATUNG   cjpeg   /usr/projects/sidgrid/usr/bin/cjpeg   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_clip_volume   /usr/projects/sidgrid/usr/bin/@clip_volume   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_CommandGlobb   /usr/projects/sidgrid/usr/bin/@CommandGlobb   INSTALLED INTEL32::LINUX null
-NCSATUNG   CompareSurfaces   /usr/projects/sidgrid/usr/bin/CompareSurfaces   INSTALLED INTEL32::LINUX null
-NCSATUNG   ConvertDset   /usr/projects/sidgrid/usr/bin/ConvertDset   INSTALLED INTEL32::LINUX null
-NCSATUNG   ConvertSurface   /usr/projects/sidgrid/usr/bin/ConvertSurface   INSTALLED INTEL32::LINUX null
-NCSATUNG   ConvexHull   /usr/projects/sidgrid/usr/bin/ConvexHull   INSTALLED INTEL32::LINUX null
-NCSATUNG   count   /usr/projects/sidgrid/usr/bin/count   INSTALLED INTEL32::LINUX null
-NCSATUNG   CreateIcosahedron   /usr/projects/sidgrid/usr/bin/CreateIcosahedron   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_db_mod_py   /usr/projects/sidgrid/usr/bin/db_mod.py   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_dicom_hdr   /usr/projects/sidgrid/usr/bin/dicom_hdr   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_dicom_to_raw   /usr/projects/sidgrid/usr/bin/dicom_to_raw   INSTALLED INTEL32::LINUX null
-NCSATUNG   Dimon   /usr/projects/sidgrid/usr/bin/Dimon   INSTALLED INTEL32::LINUX null
-NCSATUNG   djpeg   /usr/projects/sidgrid/usr/bin/djpeg   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_DoPerRoi_py   /usr/projects/sidgrid/usr/bin/@DoPerRoi.py   INSTALLED INTEL32::LINUX null
-NCSATUNG   DriveSuma   /usr/projects/sidgrid/usr/bin/DriveSuma   INSTALLED INTEL32::LINUX null
-NCSATUNG   DTIStudioFibertoSegments   /usr/projects/sidgrid/usr/bin/DTIStudioFibertoSegments   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_DTI_studio_reposition   /usr/projects/sidgrid/usr/bin/@DTI_studio_reposition   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_ent16   /usr/projects/sidgrid/usr/bin/ent16   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_FD2   /usr/projects/sidgrid/usr/bin/FD2   INSTALLED INTEL32::LINUX null
-NCSATUNG   fftest   /usr/projects/sidgrid/usr/bin/fftest   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_file_tool   /usr/projects/sidgrid/usr/bin/file_tool   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_fim2   /usr/projects/sidgrid/usr/bin/fim2   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_FindAfniDsetPath   /usr/projects/sidgrid/usr/bin/@FindAfniDsetPath   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_fix_FSsphere   /usr/projects/sidgrid/usr/bin/@fix_FSsphere   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_float_scan   /usr/projects/sidgrid/usr/bin/float_scan   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_from3d   /usr/projects/sidgrid/usr/bin/from3d   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_FromRAI   /usr/projects/sidgrid/usr/bin/@FromRAI   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_FSread_annot   /usr/projects/sidgrid/usr/bin/FSread_annot   INSTALLED INTEL32::LINUX null
-NCSATUNG   ftosh   /usr/projects/sidgrid/usr/bin/ftosh   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_ge_header   /usr/projects/sidgrid/usr/bin/ge_header   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_GetAfniOrient   /usr/projects/sidgrid/usr/bin/@GetAfniOrient   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_GetAfniPrefix   /usr/projects/sidgrid/usr/bin/@GetAfniPrefix   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_GetAfniView   /usr/projects/sidgrid/usr/bin/@GetAfniView   INSTALLED INTEL32::LINUX null
-NCSATUNG   Ifile   /usr/projects/sidgrid/usr/bin/Ifile   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_im2niml   /usr/projects/sidgrid/usr/bin/im2niml   INSTALLED INTEL32::LINUX null
-NCSATUNG   imand   /usr/projects/sidgrid/usr/bin/imand   INSTALLED INTEL32::LINUX null
-NCSATUNG   imaver   /usr/projects/sidgrid/usr/bin/imaver   INSTALLED INTEL32::LINUX null
-NCSATUNG   imcalc   /usr/projects/sidgrid/usr/bin/imcalc   INSTALLED INTEL32::LINUX null
-NCSATUNG   imcutup   /usr/projects/sidgrid/usr/bin/imcutup   INSTALLED INTEL32::LINUX null
-NCSATUNG   imdump   /usr/projects/sidgrid/usr/bin/imdump   INSTALLED INTEL32::LINUX null
-NCSATUNG   immask   /usr/projects/sidgrid/usr/bin/immask   INSTALLED INTEL32::LINUX null
-NCSATUNG   Imon   /usr/projects/sidgrid/usr/bin/Imon   INSTALLED INTEL32::LINUX null
-NCSATUNG   imreg   /usr/projects/sidgrid/usr/bin/imreg   INSTALLED INTEL32::LINUX null
-NCSATUNG   imrotate   /usr/projects/sidgrid/usr/bin/imrotate   INSTALLED INTEL32::LINUX null
-NCSATUNG   imstack   /usr/projects/sidgrid/usr/bin/imstack   INSTALLED INTEL32::LINUX null
-NCSATUNG   imstat   /usr/projects/sidgrid/usr/bin/imstat   INSTALLED INTEL32::LINUX null
-NCSATUNG   imupsam   /usr/projects/sidgrid/usr/bin/imupsam   INSTALLED INTEL32::LINUX null
-NCSATUNG   inspec   /usr/projects/sidgrid/usr/bin/inspec   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_IsoMasks   /usr/projects/sidgrid/usr/bin/@IsoMasks   INSTALLED INTEL32::LINUX null
-NCSATUNG   IsoSurface   /usr/projects/sidgrid/usr/bin/IsoSurface   INSTALLED INTEL32::LINUX null
-NCSATUNG   MakeColorMap   /usr/projects/sidgrid/usr/bin/MakeColorMap   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_make_stim_file   /usr/projects/sidgrid/usr/bin/@make_stim_file   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_make_stim_times_py   /usr/projects/sidgrid/usr/bin/make_stim_times.py   INSTALLED INTEL32::LINUX null
-NCSATUNG   MapIcosahedron   /usr/projects/sidgrid/usr/bin/MapIcosahedron   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_mayo_analyze   /usr/projects/sidgrid/usr/bin/mayo_analyze   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_mpeg_encode   /usr/projects/sidgrid/usr/bin/mpeg_encode   INSTALLED INTEL32::LINUX null
-NCSATUNG   mpegtoppm   /usr/projects/sidgrid/usr/bin/mpegtoppm   INSTALLED INTEL32::LINUX null
-NCSATUNG   mritopgm   /usr/projects/sidgrid/usr/bin/mritopgm   INSTALLED INTEL32::LINUX null
-NCSATUNG   myget   /usr/projects/sidgrid/usr/bin/myget   INSTALLED INTEL32::LINUX null
-NCSATUNG   nicat   /usr/projects/sidgrid/usr/bin/nicat   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_nifti1_test   /usr/projects/sidgrid/usr/bin/nifti1_test   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_nifti_tool   /usr/projects/sidgrid/usr/bin/nifti_tool   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_NoExt   /usr/projects/sidgrid/usr/bin/@NoExt   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_np   /usr/projects/sidgrid/usr/bin/@np   INSTALLED INTEL32::LINUX null
-NCSATUNG   nsize   /usr/projects/sidgrid/usr/bin/nsize   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_option_list_py   /usr/projects/sidgrid/usr/bin/option_list.py   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_p2t   /usr/projects/sidgrid/usr/bin/p2t   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_parse_afni_name   /usr/projects/sidgrid/usr/bin/@parse_afni_name   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_parse_name   /usr/projects/sidgrid/usr/bin/@parse_name   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_Purify_1D   /usr/projects/sidgrid/usr/bin/@Purify_1D   INSTALLED INTEL32::LINUX null
-NCSATUNG   qhull   /usr/projects/sidgrid/usr/bin/qhull   INSTALLED INTEL32::LINUX null
-NCSATUNG   quickspec   /usr/projects/sidgrid/usr/bin/quickspec   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_RenamePanga   /usr/projects/sidgrid/usr/bin/@RenamePanga   INSTALLED INTEL32::LINUX null
-NCSATUNG   rmz   /usr/projects/sidgrid/usr/bin/rmz   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_ROI2dataset   /usr/projects/sidgrid/usr/bin/ROI2dataset   INSTALLED INTEL32::LINUX null
-NCSATUNG   ROIgrow   /usr/projects/sidgrid/usr/bin/ROIgrow   INSTALLED INTEL32::LINUX null
-NCSATUNG   rotcom   /usr/projects/sidgrid/usr/bin/rotcom   INSTALLED INTEL32::LINUX null
-NCSATUNG   RSFgen   /usr/projects/sidgrid/usr/bin/RSFgen   INSTALLED INTEL32::LINUX null
-NCSATUNG   rtfeedme   /usr/projects/sidgrid/usr/bin/rtfeedme   INSTALLED INTEL32::LINUX null
-NCSATUNG   SampBias   /usr/projects/sidgrid/usr/bin/SampBias   INSTALLED INTEL32::LINUX null
-NCSATUNG   ScaleToMap   /usr/projects/sidgrid/usr/bin/ScaleToMap   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_serial_helper   /usr/projects/sidgrid/usr/bin/serial_helper   INSTALLED INTEL32::LINUX null
-NCSATUNG   sfim   /usr/projects/sidgrid/usr/bin/sfim   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_siemens_vision   /usr/projects/sidgrid/usr/bin/siemens_vision   INSTALLED INTEL32::LINUX null
-NCSATUNG   sqwave   /usr/projects/sidgrid/usr/bin/sqwave   INSTALLED INTEL32::LINUX null
-NCSATUNG   strblast   /usr/projects/sidgrid/usr/bin/strblast   INSTALLED INTEL32::LINUX null
-NCSATUNG   suma   /usr/projects/sidgrid/usr/bin/suma   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_suma_aligntoexp   /usr/projects/sidgrid/usr/bin/suma_aligntoexp   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_SUMA_AlignToExperiment   /usr/projects/sidgrid/usr/bin/SUMA_AlignToExperiment   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_suma_aligntoexp_py   /usr/projects/sidgrid/usr/bin/suma_aligntoexp.py   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_suma_change_spec   /usr/projects/sidgrid/usr/bin/suma_change_spec   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_SUMA_glxdino   /usr/projects/sidgrid/usr/bin/SUMA_glxdino   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_SUMA_Make_Spec_Caret   /usr/projects/sidgrid/usr/bin/@SUMA_Make_Spec_Caret   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_SUMA_Make_Spec_FS   /usr/projects/sidgrid/usr/bin/@SUMA_Make_Spec_FS   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_SUMA_Make_Spec_SF   /usr/projects/sidgrid/usr/bin/@SUMA_Make_Spec_SF   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_SUMA_paperplane   /usr/projects/sidgrid/usr/bin/SUMA_paperplane   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_SUMA_pixmap2eps   /usr/projects/sidgrid/usr/bin/SUMA_pixmap2eps   INSTALLED INTEL32::LINUX null
-NCSATUNG   SurfaceMetrics   /usr/projects/sidgrid/usr/bin/SurfaceMetrics   INSTALLED INTEL32::LINUX null
-NCSATUNG   SurfClust   /usr/projects/sidgrid/usr/bin/SurfClust   INSTALLED INTEL32::LINUX null
-NCSATUNG   SurfDsetInfo   /usr/projects/sidgrid/usr/bin/SurfDsetInfo   INSTALLED INTEL32::LINUX null
-NCSATUNG   SurfInfo   /usr/projects/sidgrid/usr/bin/SurfInfo   INSTALLED INTEL32::LINUX null
-NCSATUNG   SurfMeasures   /usr/projects/sidgrid/usr/bin/SurfMeasures   INSTALLED INTEL32::LINUX null
-NCSATUNG   SurfPatch   /usr/projects/sidgrid/usr/bin/SurfPatch   INSTALLED INTEL32::LINUX null
-NCSATUNG   SurfQual   /usr/projects/sidgrid/usr/bin/SurfQual   INSTALLED INTEL32::LINUX null
-NCSATUNG   SurfSmooth   /usr/projects/sidgrid/usr/bin/SurfSmooth   INSTALLED INTEL32::LINUX null
-NCSATUNG   SurfToSurf   /usr/projects/sidgrid/usr/bin/SurfToSurf   INSTALLED INTEL32::LINUX null
-NCSATUNG   tfim   /usr/projects/sidgrid/usr/bin/tfim   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_to3d   /usr/projects/sidgrid/usr/bin/to3d   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_ToRAI   /usr/projects/sidgrid/usr/bin/@ToRAI   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_UpdateAfni   /usr/projects/sidgrid/usr/bin/@UpdateAfni   INSTALLED INTEL32::LINUX null
-NCSATUNG   Vecwarp   /usr/projects/sidgrid/usr/bin/Vecwarp   INSTALLED INTEL32::LINUX null
-NCSATUNG   AFNI_VolCenter   /usr/projects/sidgrid/usr/bin/@VolCenter   INSTALLED INTEL32::LINUX null
-NCSATUNG   waver   /usr/projects/sidgrid/usr/bin/waver   INSTALLED INTEL32::LINUX null
-NCSATUNG   whereami   /usr/projects/sidgrid/usr/bin/whereami   INSTALLED INTEL32::LINUX null
-NCSATUNG   whirlgif   /usr/projects/sidgrid/usr/bin/whirlgif   INSTALLED INTEL32::LINUX null
-NCSATUNG   Xphace   /usr/projects/sidgrid/usr/bin/Xphace   INSTALLED INTEL32::LINUX null
-NCSATUNG   RInvoke /cfs/projects/sidgrid/scripts/RInvoke.sh   INSTALLED INTEL32::LINUX   GLOBUS::maxwalltime=239
-NCSATUNG   tar  /bin/tar INSTALLED INTEL32::LINUX null
-
 ####-----------------LONESTAR-----------------------------
 LONESTAR   AFNI_1dcat   /projects/tg/SIDGrid/afni_bin/1dcat   INSTALLED INTEL64::LINUX null
 LONESTAR   AFNI_1ddot   /projects/tg/SIDGrid/afni_bin/1ddot   INSTALLED INTEL64::LINUX null



From noreply at svn.ci.uchicago.edu  Tue Jul 22 03:03:57 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 22 Jul 2008 03:03:57 -0500 (CDT)
Subject: [Swift-commit] r2145 - trunk/docs
Message-ID: <20080722080357.8042B9B4001@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-22 03:03:54 -0500 (Tue, 22 Jul 2008)
New Revision: 2145

Modified:
   trunk/docs/userguide.xml
Log:
id on MPI section

Modified: trunk/docs/userguide.xml
===================================================================
--- trunk/docs/userguide.xml	2008-07-22 01:14:58 UTC (rev 2144)
+++ trunk/docs/userguide.xml	2008-07-22 08:03:54 UTC (rev 2145)
@@ -2360,7 +2360,7 @@
 </para>
 
 		</section>
-		<section><title>Launching MPI jobs from Swift</title>
+		<section id="tips.mpi"><title>Launching MPI jobs from Swift</title>
 <para>
 Here is an example of running a simple MPI program.
 </para>



From noreply at svn.ci.uchicago.edu  Thu Jul 24 16:36:25 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 24 Jul 2008 16:36:25 -0500 (CDT)
Subject: [Swift-commit] r2147 - SwiftApps/SIDGrid/config
Message-ID: <20080724213625.A5CC49B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-24 16:36:24 -0500 (Thu, 24 Jul 2008)
New Revision: 2147

Modified:
   SwiftApps/SIDGrid/config/tc.data
Log:
bigred added

Modified: SwiftApps/SIDGrid/config/tc.data
===================================================================
--- SwiftApps/SIDGrid/config/tc.data	2008-07-22 15:27:35 UTC (rev 2146)
+++ SwiftApps/SIDGrid/config/tc.data	2008-07-24 21:36:24 UTC (rev 2147)
@@ -585,7 +585,7 @@
 ANLUCTERAGRID32      reconFINAL   /soft/community/SIDGrid/freesurfer/bin/reconFINAL.sh  INSTALLED INTEL32::LINUX  env::FREESURFER_HOME=/soft/community/SIDGrid/freesurfer
 ANLUCTERAGRID32   R /soft/community/SIDGrid/R/bin/R  INSTALLED INTEL32::LINUX  null
 ANLUCTERAGRID32   RInvoke /soft/community/SIDGrid/R/bin/RInvoke.sh  INSTALLED INTEL32::LINUX  null
-ANLUCTERAGRID32   Mediator /soft/community/SIDGrid/scripts/Mediator.py  INSTALLED INTEL32::LINUX  env::PATHPREFIX=/soft/community/SIDGrid/usr/bin;GLOBUS::host_types=ia32-compute
+ANLUCTERAGRID32   Mediator /soft/community/SIDGrid/scripts/Mediator.py  INSTALLED INTEL32::LINUX  env::PATHPREFIX=/soft/community/SIDGrid/usr/bin;GLOBUS::host_types=ia32-compute,maxwalltime=300
 ANLUCTERAGRID32   Vol2Surf /soft/community/SIDGrid/abin/Vol2Surf INSTALLED INTEL32::LINUX null
 
 ####-----------------ANLUCTERAGRID64-----------------------------
@@ -865,7 +865,7 @@
 ANLUCTERAGRID64   whirlgif   /soft/community/SIDGrid/usr/bin/whirlgif   INSTALLED INTEL64::LINUX GLOBUS::maxwalltime=240;env::PATHPREFIX=/soft/community/SIDGrid/usr/bin,LD_LIBRARY_PATH=/soft/community/SIDGrid/usr/lib
 ANLUCTERAGRID64   Xphace   /soft/community/SIDGrid/usr/bin/Xphace   INSTALLED INTEL64::LINUX GLOBUS::maxwalltime=240;env::PATHPREFIX=/soft/community/SIDGrid/usr/bin,LD_LIBRARY_PATH=/soft/community/SIDGrid/usr/lib
 #-------------------------------------------------------
-ANLUCTERAGRID64   Mediator /soft/community/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX  null
+ANLUCTERAGRID64   Mediator /soft/community/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX  GLOBUS::maxwalltime=300
 ANLUCTERAGRID64   echo /bin/echo  INSTALLED INTEL64::LINUX  null
 ANLUCTERAGRID64   RInvoke /soft/community/SIDGrid/scripts/RInvoke.sh  INSTALLED INTEL64::LINUX  null
 ANLUCTERAGRID64   env /bin/env INSTALLED INTEL64::LINUX  null
@@ -1717,3 +1717,9 @@
 RANGER env /usr/bin/env  INSTALLED INTEL64::LINUX   null
 RANGER Mediator /scratch/projects/tg/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX   env::PATHPREFIX=/scratch/projects/tg/SIDGrid/usr/bin;GLOBUS::queue=development
 
+
+####---------------BIGRED----------------------------------------
+
+BIGRED env /usr/bin/env INSTALLED INTEL64::LINUX null
+BIGRED echo /bin/echo INSTALLED INTEL64::LINUX null
+BIGRED Mediator /N/u/tg-scommun/BigRed/sidgrid/scripts/Mediator.py INSTALLED INTEL64::LINUX GLOBUS::maxwalltime=120;env::PATHPREFIX=/N/u/tg-scommun/BigRed/sidgrid/usr/bin



From noreply at svn.ci.uchicago.edu  Thu Jul 24 19:00:45 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 24 Jul 2008 19:00:45 -0500 (CDT)
Subject: [Swift-commit] r2148 - SwiftApps/SIDGrid/config
Message-ID: <20080725000045.791F29B4001@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-24 19:00:44 -0500 (Thu, 24 Jul 2008)
New Revision: 2148

Modified:
   SwiftApps/SIDGrid/config/tc.data
Log:
mediator setting on ranger updtaed

Modified: SwiftApps/SIDGrid/config/tc.data
===================================================================
--- SwiftApps/SIDGrid/config/tc.data	2008-07-24 21:36:24 UTC (rev 2147)
+++ SwiftApps/SIDGrid/config/tc.data	2008-07-25 00:00:44 UTC (rev 2148)
@@ -1715,7 +1715,7 @@
 RANGER RInvoke /scratch/projects/tg/SIDGrid/scripts/RInvoke.sh  INSTALLED INTEL64::LINUX   env::PATHPREFIX=/scratch/projects/tg/SIDGrid/usr/bin
 RANGER echo /bin/echo  INSTALLED INTEL64::LINUX   null
 RANGER env /usr/bin/env  INSTALLED INTEL64::LINUX   null
-RANGER Mediator /scratch/projects/tg/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX   env::PATHPREFIX=/scratch/projects/tg/SIDGrid/usr/bin;GLOBUS::queue=development
+RANGER Mediator /scratch/projects/tg/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX   env::PATHPREFIX=/scratch/projects/tg/SIDGrid/usr/bin,LD_LIBRARY_PATH=/scratch/projects/tg/SIDGrid/usr/lib/mysql
 
 
 ####---------------BIGRED----------------------------------------



From noreply at svn.ci.uchicago.edu  Sat Jul 26 18:59:32 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sat, 26 Jul 2008 18:59:32 -0500 (CDT)
Subject: [Swift-commit] r2149 -
	provider-deef/src/org/globus/cog/abstraction/impl/execution/deef
Message-ID: <20080726235932.4556B1360015@www.ci.uchicago.edu>

Author: wilde
Date: 2008-07-26 18:59:31 -0500 (Sat, 26 Jul 2008)
New Revision: 2149

Modified:
   provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java
Log:
Set Swift server IP from cog property for case of multiple interfaces, and create one resource pool per Falkon server

Modified: provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java
===================================================================
--- provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java	2008-07-25 00:00:44 UTC (rev 2148)
+++ provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java	2008-07-26 23:59:31 UTC (rev 2149)
@@ -37,6 +37,9 @@
 import org.globus.cog.abstraction.impl.common.task.InvalidServiceContactException;
 import org.globus.cog.abstraction.impl.common.task.TaskSubmissionException;
 
+import org.globus.common.CoGProperties;
+
+
 public class ResourcePool {
     static Logger logger = Logger.getLogger(ResourcePool.class.getName());
     private EndpointReferenceType factoryEPR = null;
@@ -49,7 +52,8 @@
     private String server = "";
     private int num = 1;
     private static int crtResource = 0;
-    private static ResourcePool rp = null;
+    // private static ResourcePool rp = null;
+    private static HashMap rpmap = null;
     private Map tasks;
     private Notification userNot = null;
     private NotificationThread notThread;
@@ -60,8 +64,14 @@
     private UserJob job = null;
 
     public static synchronized ResourcePool instance(String server, int num) throws InvalidServiceContactException {
+        ResourcePool rp = null;
+
+        if( rpmap == null )
+            rpmap = new HashMap();
+        rp = (ResourcePool) rpmap.get(server);
 	if (rp == null) {
 	    rp = new ResourcePool();
+            rpmap.put(server, rp);
 	    rp.server = server;
 	    rp.num = num;
 	    rp.tasks = Collections.synchronizedMap(new HashMap());
@@ -166,13 +176,13 @@
          }
     }
 
-    public EndpointReferenceType getNextResource() {
+    public synchronized EndpointReferenceType getNextResource() {
 	int next = crtResource;
 	crtResource = (crtResource + 1) % num;
 	return (EndpointReferenceType) eprPool.get(next);
     }
 
-    public GPPortType getNextResourcePort() {
+    public synchronized GPPortType getNextResourcePort() {
 	int next = crtResource;
 	crtResource = (crtResource + 1) % num;
 	return (GPPortType) gptPool.get(next);
@@ -208,7 +218,7 @@
 	*/
     }
 
-    public static String getMachNamePort(Notification userNot){
+    public static String getMachNamePort__ORIG__(Notification userNot){
         String machName = "";
         //String machIP;
         try {
@@ -223,6 +233,38 @@
 	return machName;
     }
 
+/**** From Ioan:
+   public String getMachNamePort__IOANS_EMAIL__(int recvPort)
+   {
+       String machName = new String("localhost:"+recvPort);
+       try
+       {
+           String method = new String("overide");
+           if (NOTIFICATION_ENDPOINT == null)
+           {
+               NOTIFICATION_ENDPOINT = java.net.InetAddress.getLocalHost().getCanonicalHostName();
+               method = new String("automatic");
+           }
+           machName = NOTIFICATION_ENDPOINT +  ":" + recvPort;
+           System.out.println("Notification Endpoint (" + method + "): " + NOTIFICATION_ENDPOINT);
+       }
+       catch (Exception e)
+       {
+           System.out.println("WORKER: ERROR: java.net.InetAddress.getLocalHost().getHostName() failed " + e);
+           if (DEBUG) e.printStackTrace();
+       }
+       return machName;
+   }
+****/
+
+    public static String getMachNamePort(Notification userNot){
+        //String machIP = VDL2Config.getIP();
+	String machIP = CoGProperties.getDefault().getIPAddress();
+        String machNamePort = new String (machIP +  ":" + userNot.recvPort);
+        logger.debug("WORKER: Machine ID = " + machNamePort);
+	return machNamePort;
+    }
+
     public int getExecQueueSize() {
 	return execQueue.size();
     }



From benc at hawaga.org.uk  Sun Jul 27 05:03:49 2008
From: benc at hawaga.org.uk (Ben Clifford)
Date: Sun, 27 Jul 2008 10:03:49 +0000 (GMT)
Subject: [Swift-commit] r2149 -
	provider-deef/src/org/globus/cog/abstraction/impl/execution/deef
In-Reply-To: <20080726235932.4556B1360015@www.ci.uchicago.edu>
References: <20080726235932.4556B1360015@www.ci.uchicago.edu>
Message-ID: <Pine.LNX.4.64.0807271002490.5076@dildano.hawaga.org.uk>



did you really write this code or is it from somone else?

On Sat, 26 Jul 2008, noreply at www.ci.uchicago.edu wrote:

> Author: wilde
> Date: 2008-07-26 18:59:31 -0500 (Sat, 26 Jul 2008)
> New Revision: 2149
> 
> Modified:
>    provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java
> Log:
> Set Swift server IP from cog property for case of multiple interfaces, and create one resource pool per Falkon server
> 
> Modified: provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java
> ===================================================================
> --- provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java	2008-07-25 00:00:44 UTC (rev 2148)
> +++ provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java	2008-07-26 23:59:31 UTC (rev 2149)
> @@ -37,6 +37,9 @@
>  import org.globus.cog.abstraction.impl.common.task.InvalidServiceContactException;
>  import org.globus.cog.abstraction.impl.common.task.TaskSubmissionException;
>  
> +import org.globus.common.CoGProperties;
> +
> +
>  public class ResourcePool {
>      static Logger logger = Logger.getLogger(ResourcePool.class.getName());
>      private EndpointReferenceType factoryEPR = null;
> @@ -49,7 +52,8 @@
>      private String server = "";
>      private int num = 1;
>      private static int crtResource = 0;
> -    private static ResourcePool rp = null;
> +    // private static ResourcePool rp = null;
> +    private static HashMap rpmap = null;
>      private Map tasks;
>      private Notification userNot = null;
>      private NotificationThread notThread;
> @@ -60,8 +64,14 @@
>      private UserJob job = null;
>  
>      public static synchronized ResourcePool instance(String server, int num) throws InvalidServiceContactException {
> +        ResourcePool rp = null;
> +
> +        if( rpmap == null )
> +            rpmap = new HashMap();
> +        rp = (ResourcePool) rpmap.get(server);
>  	if (rp == null) {
>  	    rp = new ResourcePool();
> +            rpmap.put(server, rp);
>  	    rp.server = server;
>  	    rp.num = num;
>  	    rp.tasks = Collections.synchronizedMap(new HashMap());
> @@ -166,13 +176,13 @@
>           }
>      }
>  
> -    public EndpointReferenceType getNextResource() {
> +    public synchronized EndpointReferenceType getNextResource() {
>  	int next = crtResource;
>  	crtResource = (crtResource + 1) % num;
>  	return (EndpointReferenceType) eprPool.get(next);
>      }
>  
> -    public GPPortType getNextResourcePort() {
> +    public synchronized GPPortType getNextResourcePort() {
>  	int next = crtResource;
>  	crtResource = (crtResource + 1) % num;
>  	return (GPPortType) gptPool.get(next);
> @@ -208,7 +218,7 @@
>  	*/
>      }
>  
> -    public static String getMachNamePort(Notification userNot){
> +    public static String getMachNamePort__ORIG__(Notification userNot){
>          String machName = "";
>          //String machIP;
>          try {
> @@ -223,6 +233,38 @@
>  	return machName;
>      }
>  
> +/**** From Ioan:
> +   public String getMachNamePort__IOANS_EMAIL__(int recvPort)
> +   {
> +       String machName = new String("localhost:"+recvPort);
> +       try
> +       {
> +           String method = new String("overide");
> +           if (NOTIFICATION_ENDPOINT == null)
> +           {
> +               NOTIFICATION_ENDPOINT = java.net.InetAddress.getLocalHost().getCanonicalHostName();
> +               method = new String("automatic");
> +           }
> +           machName = NOTIFICATION_ENDPOINT +  ":" + recvPort;
> +           System.out.println("Notification Endpoint (" + method + "): " + NOTIFICATION_ENDPOINT);
> +       }
> +       catch (Exception e)
> +       {
> +           System.out.println("WORKER: ERROR: java.net.InetAddress.getLocalHost().getHostName() failed " + e);
> +           if (DEBUG) e.printStackTrace();
> +       }
> +       return machName;
> +   }
> +****/
> +
> +    public static String getMachNamePort(Notification userNot){
> +        //String machIP = VDL2Config.getIP();
> +	String machIP = CoGProperties.getDefault().getIPAddress();
> +        String machNamePort = new String (machIP +  ":" + userNot.recvPort);
> +        logger.debug("WORKER: Machine ID = " + machNamePort);
> +	return machNamePort;
> +    }
> +
>      public int getExecQueueSize() {
>  	return execQueue.size();
>      }
> 
> _______________________________________________
> Swift-commit mailing list
> Swift-commit at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-commit
> 
> 


From wilde at mcs.anl.gov  Sun Jul 27 11:40:28 2008
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Sun, 27 Jul 2008 11:40:28 -0500
Subject: [Swift-commit] r2149 -
	provider-deef/src/org/globus/cog/abstraction/impl/execution/deef
In-Reply-To: <Pine.LNX.4.64.0807271002490.5076@dildano.hawaga.org.uk>
References: <20080726235932.4556B1360015@www.ci.uchicago.edu>
	<Pine.LNX.4.64.0807271002490.5076@dildano.hawaga.org.uk>
Message-ID: <488CA4FC.7090401@mcs.anl.gov>

I wrote it but forgot to remove junk-notes before I committed.
I will do that.

I inserted the IP address stuff based on reading Ioan's standalone 
Falkon client code. I did the per-service Resource fix based on advice 
from Mihael.

- Mike

On 7/27/08 5:03 AM, Ben Clifford wrote:
> 
> did you really write this code or is it from somone else?
> 
> On Sat, 26 Jul 2008, noreply at www.ci.uchicago.edu wrote:
> 
>> Author: wilde
>> Date: 2008-07-26 18:59:31 -0500 (Sat, 26 Jul 2008)
>> New Revision: 2149
>>
>> Modified:
>>    provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java
>> Log:
>> Set Swift server IP from cog property for case of multiple interfaces, and create one resource pool per Falkon server
>>
>> Modified: provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java
>> ===================================================================
>> --- provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java	2008-07-25 00:00:44 UTC (rev 2148)
>> +++ provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java	2008-07-26 23:59:31 UTC (rev 2149)
>> @@ -37,6 +37,9 @@
>>  import org.globus.cog.abstraction.impl.common.task.InvalidServiceContactException;
>>  import org.globus.cog.abstraction.impl.common.task.TaskSubmissionException;
>>  
>> +import org.globus.common.CoGProperties;
>> +
>> +
>>  public class ResourcePool {
>>      static Logger logger = Logger.getLogger(ResourcePool.class.getName());
>>      private EndpointReferenceType factoryEPR = null;
>> @@ -49,7 +52,8 @@
>>      private String server = "";
>>      private int num = 1;
>>      private static int crtResource = 0;
>> -    private static ResourcePool rp = null;
>> +    // private static ResourcePool rp = null;
>> +    private static HashMap rpmap = null;
>>      private Map tasks;
>>      private Notification userNot = null;
>>      private NotificationThread notThread;
>> @@ -60,8 +64,14 @@
>>      private UserJob job = null;
>>  
>>      public static synchronized ResourcePool instance(String server, int num) throws InvalidServiceContactException {
>> +        ResourcePool rp = null;
>> +
>> +        if( rpmap == null )
>> +            rpmap = new HashMap();
>> +        rp = (ResourcePool) rpmap.get(server);
>>  	if (rp == null) {
>>  	    rp = new ResourcePool();
>> +            rpmap.put(server, rp);
>>  	    rp.server = server;
>>  	    rp.num = num;
>>  	    rp.tasks = Collections.synchronizedMap(new HashMap());
>> @@ -166,13 +176,13 @@
>>           }
>>      }
>>  
>> -    public EndpointReferenceType getNextResource() {
>> +    public synchronized EndpointReferenceType getNextResource() {
>>  	int next = crtResource;
>>  	crtResource = (crtResource + 1) % num;
>>  	return (EndpointReferenceType) eprPool.get(next);
>>      }
>>  
>> -    public GPPortType getNextResourcePort() {
>> +    public synchronized GPPortType getNextResourcePort() {
>>  	int next = crtResource;
>>  	crtResource = (crtResource + 1) % num;
>>  	return (GPPortType) gptPool.get(next);
>> @@ -208,7 +218,7 @@
>>  	*/
>>      }
>>  
>> -    public static String getMachNamePort(Notification userNot){
>> +    public static String getMachNamePort__ORIG__(Notification userNot){
>>          String machName = "";
>>          //String machIP;
>>          try {
>> @@ -223,6 +233,38 @@
>>  	return machName;
>>      }
>>  
>> +/**** From Ioan:
>> +   public String getMachNamePort__IOANS_EMAIL__(int recvPort)
>> +   {
>> +       String machName = new String("localhost:"+recvPort);
>> +       try
>> +       {
>> +           String method = new String("overide");
>> +           if (NOTIFICATION_ENDPOINT == null)
>> +           {
>> +               NOTIFICATION_ENDPOINT = java.net.InetAddress.getLocalHost().getCanonicalHostName();
>> +               method = new String("automatic");
>> +           }
>> +           machName = NOTIFICATION_ENDPOINT +  ":" + recvPort;
>> +           System.out.println("Notification Endpoint (" + method + "): " + NOTIFICATION_ENDPOINT);
>> +       }
>> +       catch (Exception e)
>> +       {
>> +           System.out.println("WORKER: ERROR: java.net.InetAddress.getLocalHost().getHostName() failed " + e);
>> +           if (DEBUG) e.printStackTrace();
>> +       }
>> +       return machName;
>> +   }
>> +****/
>> +
>> +    public static String getMachNamePort(Notification userNot){
>> +        //String machIP = VDL2Config.getIP();
>> +	String machIP = CoGProperties.getDefault().getIPAddress();
>> +        String machNamePort = new String (machIP +  ":" + userNot.recvPort);
>> +        logger.debug("WORKER: Machine ID = " + machNamePort);
>> +	return machNamePort;
>> +    }
>> +
>>      public int getExecQueueSize() {
>>  	return execQueue.size();
>>      }
>>
>> _______________________________________________
>> Swift-commit mailing list
>> Swift-commit at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-commit
>>
>>


From noreply at svn.ci.uchicago.edu  Sun Jul 27 13:39:45 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sun, 27 Jul 2008 13:39:45 -0500 (CDT)
Subject: [Swift-commit] r2150 - trunk/tests/misc
Message-ID: <20080727183945.8A8D61360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-27 13:39:44 -0500 (Sun, 27 Jul 2008)
New Revision: 2150

Added:
   trunk/tests/misc/external-mapper-args.args.expected
   trunk/tests/misc/external-mapper-args.mapper
   trunk/tests/misc/external-mapper-args.sh
   trunk/tests/misc/external-mapper-args.swift
Modified:
   trunk/tests/misc/run
Log:
external mapper was passing on a few parameters that it should not have been

Added: trunk/tests/misc/external-mapper-args.args.expected
===================================================================
--- trunk/tests/misc/external-mapper-args.args.expected	                        (rev 0)
+++ trunk/tests/misc/external-mapper-args.args.expected	2008-07-27 18:39:44 UTC (rev 2150)
@@ -0,0 +1 @@
+-foo bar

Added: trunk/tests/misc/external-mapper-args.mapper
===================================================================
--- trunk/tests/misc/external-mapper-args.mapper	                        (rev 0)
+++ trunk/tests/misc/external-mapper-args.mapper	2008-07-27 18:39:44 UTC (rev 2150)
@@ -0,0 +1,6 @@
+#!/bin/bash
+echo $@ > external-mapper-args.args
+echo "[0].l a.tmp"
+echo "[1].r b.tmp"
+echo "[2].l c.tmp"
+


Property changes on: trunk/tests/misc/external-mapper-args.mapper
___________________________________________________________________
Name: svn:executable
   + *

Added: trunk/tests/misc/external-mapper-args.sh
===================================================================
--- trunk/tests/misc/external-mapper-args.sh	                        (rev 0)
+++ trunk/tests/misc/external-mapper-args.sh	2008-07-27 18:39:44 UTC (rev 2150)
@@ -0,0 +1,15 @@
+#!/bin/bash
+
+rm -f external-mapper-args.args
+swift external-mapper-args.swift
+if [ "$?" != "0" ] ; then
+  echo FAILED - SwiftScript did not complete successfully
+  exit 1
+fi
+
+diff external-mapper-args.args external-mapper-args.args.expected
+
+if [ "$?" != "0" ] ; then
+  echo FAILED - external mapper received wrong arguments
+  exit 2
+fi


Property changes on: trunk/tests/misc/external-mapper-args.sh
___________________________________________________________________
Name: svn:executable
   + *

Added: trunk/tests/misc/external-mapper-args.swift
===================================================================
--- trunk/tests/misc/external-mapper-args.swift	                        (rev 0)
+++ trunk/tests/misc/external-mapper-args.swift	2008-07-27 18:39:44 UTC (rev 2150)
@@ -0,0 +1,19 @@
+type messagefile;
+
+type cols {
+  messagefile l;
+  messagefile r;
+}
+
+(messagefile t) write(string s) { 
+    app {
+        echo s stdout=@filename(t);
+    }
+}
+
+cols outfile[] <ext; exec="./external-mapper-args.mapper", foo="bar">;
+
+outfile[0].l = write("slot 0");
+outfile[1].r = write("slot 1");
+outfile[2].l = write("slot 2");
+

Modified: trunk/tests/misc/run
===================================================================
--- trunk/tests/misc/run	2008-07-26 23:59:31 UTC (rev 2149)
+++ trunk/tests/misc/run	2008-07-27 18:39:44 UTC (rev 2150)
@@ -1,6 +1,7 @@
 #!/bin/sh
 for a in clusters no-retries dryrun typecheck path-prefix restart restart2 restart3 restart4 restart5 restart-iterate workernode-local \
 ordering-extern-notlazy restart-extern ordering-extern \
+external-mapper-args \
 ; do
  ./${a}.sh
  R=$?



From noreply at svn.ci.uchicago.edu  Sun Jul 27 13:43:52 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Sun, 27 Jul 2008 13:43:52 -0500 (CDT)
Subject: [Swift-commit] r2151 - trunk/src/org/griphyn/vdl/mapping/file
Message-ID: <20080727184352.0D8F41360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-27 13:43:51 -0500 (Sun, 27 Jul 2008)
New Revision: 2151

Modified:
   trunk/src/org/griphyn/vdl/mapping/file/ExternalMapper.java
Log:
external mapper was passing on a few parameters that it should not have been

Modified: trunk/src/org/griphyn/vdl/mapping/file/ExternalMapper.java
===================================================================
--- trunk/src/org/griphyn/vdl/mapping/file/ExternalMapper.java	2008-07-27 18:39:44 UTC (rev 2150)
+++ trunk/src/org/griphyn/vdl/mapping/file/ExternalMapper.java	2008-07-27 18:43:51 UTC (rev 2151)
@@ -38,6 +38,8 @@
 		ignored.add("dbgname");
 		ignored.add("descriptor");
 		ignored.add("#basedir");
+		ignored.add("waitfor");
+		ignored.add("swift#restartid");
 	}
 
 	private static final String[] STRING_ARRAY = new String[0];



From noreply at svn.ci.uchicago.edu  Mon Jul 28 01:16:18 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 28 Jul 2008 01:16:18 -0500 (CDT)
Subject: [Swift-commit] r2152 - in trunk/tests: misc sites sites/wonky
Message-ID: <20080728061618.C8C701AB0193@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-28 01:16:17 -0500 (Mon, 28 Jul 2008)
New Revision: 2152

Added:
   trunk/tests/misc/wonky.sh
   trunk/tests/sites/wonky/
   trunk/tests/sites/wonky/wonky-90percent.xml
   trunk/tests/sites/wonky/wonky-good.xml
Log:
test to use with provider wonky

Added: trunk/tests/misc/wonky.sh
===================================================================
--- trunk/tests/misc/wonky.sh	                        (rev 0)
+++ trunk/tests/misc/wonky.sh	2008-07-28 06:16:17 UTC (rev 2152)
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+cd ../language-behaviour
+
+./generate-tc.data
+
+cat tc.data ../sites/tc.data > tmp.tc.data.sites
+
+SITE=wonky/wonky-90percent.xml
+
+echo testing site configuration: $SITE
+
+export SWIFT_TEST_PARAMS="-sites.file ../sites/${SITE} -tc.file tmp.tc.data.sites"
+
+./run 130-fmri 066-many
+


Property changes on: trunk/tests/misc/wonky.sh
___________________________________________________________________
Name: svn:executable
   + *

Added: trunk/tests/sites/wonky/wonky-90percent.xml
===================================================================
--- trunk/tests/sites/wonky/wonky-90percent.xml	                        (rev 0)
+++ trunk/tests/sites/wonky/wonky-90percent.xml	2008-07-28 06:16:17 UTC (rev 2152)
@@ -0,0 +1,10 @@
+<config>
+
+  <pool handle="localhost">
+    <gridftp  url="local://localhost" />
+    <execution provider="wonky" url="good/activefail=0.1/activedelay=0" />
+    <workdirectory >/var/tmp</workdirectory>
+    <profile namespace="karajan" key="jobThrottle">0</profile>
+  </pool>
+
+</config>

Added: trunk/tests/sites/wonky/wonky-good.xml
===================================================================
--- trunk/tests/sites/wonky/wonky-good.xml	                        (rev 0)
+++ trunk/tests/sites/wonky/wonky-good.xml	2008-07-28 06:16:17 UTC (rev 2152)
@@ -0,0 +1,10 @@
+<config>
+
+  <pool handle="localhost">
+    <gridftp  url="local://localhost" />
+    <execution provider="wonky" url="good/activefail=0/activedelay=5" />
+    <workdirectory >/var/tmp</workdirectory>
+    <profile namespace="karajan" key="jobThrottle">0</profile>
+  </pool>
+
+</config>



From noreply at svn.ci.uchicago.edu  Mon Jul 28 01:57:56 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Mon, 28 Jul 2008 01:57:56 -0500 (CDT)
Subject: [Swift-commit] r2153 - in trunk/tests: misc sites/wonky
Message-ID: <20080728065756.803E11AB0193@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-28 01:57:52 -0500 (Mon, 28 Jul 2008)
New Revision: 2153

Added:
   trunk/tests/misc/wonky80.sh
   trunk/tests/sites/wonky/wonky-80percent.xml
Log:
wonky provider test with lower probability of success. At time of commit, this test eventually hangs, unlike the other wonky test committed in r2152.

Added: trunk/tests/misc/wonky80.sh
===================================================================
--- trunk/tests/misc/wonky80.sh	                        (rev 0)
+++ trunk/tests/misc/wonky80.sh	2008-07-28 06:57:52 UTC (rev 2153)
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+cd ../language-behaviour
+
+./generate-tc.data
+
+cat tc.data ../sites/tc.data > tmp.tc.data.sites
+
+SITE=wonky/wonky-80percent.xml
+
+echo testing site configuration: $SITE
+
+export SWIFT_TEST_PARAMS="-sites.file ../sites/${SITE} -tc.file tmp.tc.data.sites"
+
+./run 130-fmri 066-many
+


Property changes on: trunk/tests/misc/wonky80.sh
___________________________________________________________________
Name: svn:executable
   + *

Added: trunk/tests/sites/wonky/wonky-80percent.xml
===================================================================
--- trunk/tests/sites/wonky/wonky-80percent.xml	                        (rev 0)
+++ trunk/tests/sites/wonky/wonky-80percent.xml	2008-07-28 06:57:52 UTC (rev 2153)
@@ -0,0 +1,10 @@
+<config>
+
+  <pool handle="localhost">
+    <gridftp  url="local://localhost" />
+    <execution provider="wonky" url="good/activefail=0.2/activedelay=0" />
+    <workdirectory >/var/tmp</workdirectory>
+    <profile namespace="karajan" key="jobThrottle">0</profile>
+  </pool>
+
+</config>



From noreply at svn.ci.uchicago.edu  Tue Jul 29 02:29:01 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 29 Jul 2008 02:29:01 -0500 (CDT)
Subject: [Swift-commit] r2154 - trunk/tests/language-behaviour
Message-ID: <20080729072901.A15F31360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-29 02:29:00 -0500 (Tue, 29 Jul 2008)
New Revision: 2154

Added:
   trunk/tests/language-behaviour/0756-ext-mapper-slow.sh
   trunk/tests/language-behaviour/0756-ext-mapper-slow.swift
   trunk/tests/language-behaviour/0757-ext-mapper-array.sh
   trunk/tests/language-behaviour/0757-ext-mapper-array.swift
Log:
two more tests for the external mapper


Added: trunk/tests/language-behaviour/0756-ext-mapper-slow.sh
===================================================================
--- trunk/tests/language-behaviour/0756-ext-mapper-slow.sh	                        (rev 0)
+++ trunk/tests/language-behaviour/0756-ext-mapper-slow.sh	2008-07-29 07:29:00 UTC (rev 2154)
@@ -0,0 +1,8 @@
+#!/bin/bash
+sleep 10s
+echo "[0].l 0756-ext-mapper.AAA.out"
+sleep 10s
+echo "[1].r 0756-ext-mapper.0.3.2.1.out"
+sleep 10s
+echo "[2].l 0756-ext-mapper.____.out"
+sleep 10s


Property changes on: trunk/tests/language-behaviour/0756-ext-mapper-slow.sh
___________________________________________________________________
Name: svn:executable
   + *

Added: trunk/tests/language-behaviour/0756-ext-mapper-slow.swift
===================================================================
--- trunk/tests/language-behaviour/0756-ext-mapper-slow.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/0756-ext-mapper-slow.swift	2008-07-29 07:29:00 UTC (rev 2154)
@@ -0,0 +1,19 @@
+type messagefile;
+
+type cols {
+  messagefile l;
+  messagefile r;
+}
+
+(messagefile t) write(string s) { 
+    app {
+        echo s stdout=@filename(t);
+    }
+}
+
+cols outfile[] <ext; exec="./0756-ext-mapper-slow.sh">;
+
+outfile[0].l = write("slot 0");
+outfile[1].r = write("slot 1");
+outfile[2].l = write("slot 2");
+

Added: trunk/tests/language-behaviour/0757-ext-mapper-array.sh
===================================================================
--- trunk/tests/language-behaviour/0757-ext-mapper-array.sh	                        (rev 0)
+++ trunk/tests/language-behaviour/0757-ext-mapper-array.sh	2008-07-29 07:29:00 UTC (rev 2154)
@@ -0,0 +1,4 @@
+echo "[0] 0757-ext-mapper-array.AAA.out"
+echo "[1] 0757-ext-mapper-array.0.3.2.1.out"
+echo "[2] 0757-ext-mapper-array.____.out"
+


Property changes on: trunk/tests/language-behaviour/0757-ext-mapper-array.sh
___________________________________________________________________
Name: svn:executable
   + *

Added: trunk/tests/language-behaviour/0757-ext-mapper-array.swift
===================================================================
--- trunk/tests/language-behaviour/0757-ext-mapper-array.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/0757-ext-mapper-array.swift	2008-07-29 07:29:00 UTC (rev 2154)
@@ -0,0 +1,14 @@
+type messagefile;
+
+(messagefile t) write(string s) { 
+    app {
+        echo s stdout=@filename(t);
+    }
+}
+
+messagefile outfile[] <ext; exec="./0757-ext-mapper-array.sh">;
+
+outfile[0] = write("slot 0");
+outfile[1] = write("slot 1");
+outfile[2] = write("slot 2");
+



From noreply at svn.ci.uchicago.edu  Tue Jul 29 02:49:23 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 29 Jul 2008 02:49:23 -0500 (CDT)
Subject: [Swift-commit] r2155 - in trunk: src/org/griphyn/vdl/mapping/file
	tests/language-behaviour tests/sites tests/sites/coaster
Message-ID: <20080729074923.7FF3D1360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-29 02:49:23 -0500 (Tue, 29 Jul 2008)
New Revision: 2155

Added:
   trunk/tests/language-behaviour/0758-ext-mapper-array.sh
   trunk/tests/language-behaviour/0758-ext-mapper-array.swift
   trunk/tests/language-behaviour/0759-ext-mapper-array.sh
   trunk/tests/language-behaviour/0759-ext-mapper-array.swift
Modified:
   trunk/src/org/griphyn/vdl/mapping/file/ExternalMapper.java
   trunk/tests/sites/coaster/coaster-local.xml
   trunk/tests/sites/run-site
Log:
make external mapper static; tests for behaviour that now works

Modified: trunk/src/org/griphyn/vdl/mapping/file/ExternalMapper.java
===================================================================
--- trunk/src/org/griphyn/vdl/mapping/file/ExternalMapper.java	2008-07-29 07:29:00 UTC (rev 2154)
+++ trunk/src/org/griphyn/vdl/mapping/file/ExternalMapper.java	2008-07-29 07:49:23 UTC (rev 2155)
@@ -139,6 +139,6 @@
 	}
 
 	public boolean isStatic() {
-		return false;
+		return true;
 	}
 }

Added: trunk/tests/language-behaviour/0758-ext-mapper-array.sh
===================================================================
--- trunk/tests/language-behaviour/0758-ext-mapper-array.sh	                        (rev 0)
+++ trunk/tests/language-behaviour/0758-ext-mapper-array.sh	2008-07-29 07:49:23 UTC (rev 2155)
@@ -0,0 +1,4 @@
+echo "[0] 0757-ext-mapper-array.AAA.out"
+echo "[1] 0757-ext-mapper-array.0.3.2.1.out"
+echo "[2] 0757-ext-mapper-array.____.out"
+


Property changes on: trunk/tests/language-behaviour/0758-ext-mapper-array.sh
___________________________________________________________________
Name: svn:executable
   + *

Added: trunk/tests/language-behaviour/0758-ext-mapper-array.swift
===================================================================
--- trunk/tests/language-behaviour/0758-ext-mapper-array.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/0758-ext-mapper-array.swift	2008-07-29 07:49:23 UTC (rev 2155)
@@ -0,0 +1,13 @@
+type messagefile;
+
+(messagefile t) write(string s) { 
+    app {
+        echo s stdout=@filename(t);
+    }
+}
+
+messagefile outfile[] <ext; exec="./0758-ext-mapper-array.sh">;
+
+foreach f,i in [0:2] {
+ outfile[i] = write("slot X");
+}

Added: trunk/tests/language-behaviour/0759-ext-mapper-array.sh
===================================================================
--- trunk/tests/language-behaviour/0759-ext-mapper-array.sh	                        (rev 0)
+++ trunk/tests/language-behaviour/0759-ext-mapper-array.sh	2008-07-29 07:49:23 UTC (rev 2155)
@@ -0,0 +1,3 @@
+#!/bin/bash
+echo [0] 0759-ext-mapper-array.a.out
+echo [1] 0759-ext-mapper-array.q.out


Property changes on: trunk/tests/language-behaviour/0759-ext-mapper-array.sh
___________________________________________________________________
Name: svn:executable
   + *

Added: trunk/tests/language-behaviour/0759-ext-mapper-array.swift
===================================================================
--- trunk/tests/language-behaviour/0759-ext-mapper-array.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/0759-ext-mapper-array.swift	2008-07-29 07:49:23 UTC (rev 2155)
@@ -0,0 +1,12 @@
+type file;
+
+(file t[]) write() { 
+    app {
+        touch "0759-ext-mapper-array.a.out" "0759-ext-mapper-array.q.out";
+    }
+}
+
+file outfile[] <ext; exec="./0759-ext-mapper-array.sh">;
+
+outfile = write();
+

Modified: trunk/tests/sites/coaster/coaster-local.xml
===================================================================
--- trunk/tests/sites/coaster/coaster-local.xml	2008-07-29 07:29:00 UTC (rev 2154)
+++ trunk/tests/sites/coaster/coaster-local.xml	2008-07-29 07:49:23 UTC (rev 2155)
@@ -15,6 +15,7 @@
     <gridftp  url="local://localhost" />
     <execution provider="coaster" jobmanager="local:local" url="localhost" />
     <workdirectory >/var/tmp</workdirectory>
+    <profile namespace="globus" key="coastersPerNode">5</profile>
   </pool>
 
 </config>

Modified: trunk/tests/sites/run-site
===================================================================
--- trunk/tests/sites/run-site	2008-07-29 07:29:00 UTC (rev 2154)
+++ trunk/tests/sites/run-site	2008-07-29 07:49:23 UTC (rev 2155)
@@ -12,5 +12,5 @@
 
 export SWIFT_TEST_PARAMS="-sites.file ../sites/${SITE} -tc.file tmp.tc.data.sites"
 
-./run 061-cattwo 130-fmri 103-quote.swift 1032-singlequote.swift 1031-quote.swift 1033-singlequote.swift 141-space-in-filename 142-space-and-quotes
+./run 061-cattwo 0651-several-delay.swift 130-fmri 103-quote.swift 1032-singlequote.swift 1031-quote.swift 1033-singlequote.swift 141-space-in-filename 142-space-and-quotes
 #./run 061-cattwo



From noreply at svn.ci.uchicago.edu  Tue Jul 29 02:55:57 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 29 Jul 2008 02:55:57 -0500 (CDT)
Subject: [Swift-commit] r2156 - trunk/tests/sites
Message-ID: <20080729075557.344971360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-29 02:55:56 -0500 (Tue, 29 Jul 2008)
New Revision: 2156

Modified:
   trunk/tests/sites/run-site
Log:
r2155 committed a change to this file that should not have been committed

Modified: trunk/tests/sites/run-site
===================================================================
--- trunk/tests/sites/run-site	2008-07-29 07:49:23 UTC (rev 2155)
+++ trunk/tests/sites/run-site	2008-07-29 07:55:56 UTC (rev 2156)
@@ -12,5 +12,5 @@
 
 export SWIFT_TEST_PARAMS="-sites.file ../sites/${SITE} -tc.file tmp.tc.data.sites"
 
-./run 061-cattwo 0651-several-delay.swift 130-fmri 103-quote.swift 1032-singlequote.swift 1031-quote.swift 1033-singlequote.swift 141-space-in-filename 142-space-and-quotes
+./run 061-cattwo 130-fmri 103-quote.swift 1032-singlequote.swift 1031-quote.swift 1033-singlequote.swift 141-space-in-filename 142-space-and-quotes
 #./run 061-cattwo



From noreply at svn.ci.uchicago.edu  Tue Jul 29 03:00:13 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 29 Jul 2008 03:00:13 -0500 (CDT)
Subject: [Swift-commit] r2157 - trunk/tests/language-behaviour
Message-ID: <20080729080013.B45561360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-29 03:00:11 -0500 (Tue, 29 Jul 2008)
New Revision: 2157

Added:
   trunk/tests/language-behaviour/0651-several-delay.swift
   trunk/tests/language-behaviour/066-many.swift
Log:
tests that run many jobs and longer jobs

Added: trunk/tests/language-behaviour/0651-several-delay.swift
===================================================================
--- trunk/tests/language-behaviour/0651-several-delay.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/0651-several-delay.swift	2008-07-29 08:00:11 UTC (rev 2157)
@@ -0,0 +1,10 @@
+
+delay() { 
+    app {
+        sleep "90s";
+    }
+}
+
+foreach i in [1:10] {
+delay();
+}

Added: trunk/tests/language-behaviour/066-many.swift
===================================================================
--- trunk/tests/language-behaviour/066-many.swift	                        (rev 0)
+++ trunk/tests/language-behaviour/066-many.swift	2008-07-29 08:00:11 UTC (rev 2157)
@@ -0,0 +1,10 @@
+
+p() { 
+    app {
+        touch "nop";
+    }
+}
+
+foreach i in [1:1000] {
+    p();
+}



From noreply at svn.ci.uchicago.edu  Tue Jul 29 05:30:41 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Tue, 29 Jul 2008 05:30:41 -0500 (CDT)
Subject: [Swift-commit] r2159 -
	provider-wonky/src/org/globus/cog/abstraction/impl/execution/wonky
Message-ID: <20080729103041.CE0F01360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-29 05:30:40 -0500 (Tue, 29 Jul 2008)
New Revision: 2159

Modified:
   provider-wonky/src/org/globus/cog/abstraction/impl/execution/wonky/JobSubmissionTaskHandler.java
Log:
tidy up output some

Modified: provider-wonky/src/org/globus/cog/abstraction/impl/execution/wonky/JobSubmissionTaskHandler.java
===================================================================
--- provider-wonky/src/org/globus/cog/abstraction/impl/execution/wonky/JobSubmissionTaskHandler.java	2008-07-29 09:06:10 UTC (rev 2158)
+++ provider-wonky/src/org/globus/cog/abstraction/impl/execution/wonky/JobSubmissionTaskHandler.java	2008-07-29 10:30:40 UTC (rev 2159)
@@ -244,14 +244,12 @@
             if (killed) {
                 return;
             }
-System.err.println("Wonky job completed with exitCode "+exitCode);
             if (exitCode == 0) {
                 if(failDelay("completed")) {
                     this.task.setStatus(Status.COMPLETED);
                 } else {
                     this.task.setStatus(Status.FAILED);
                 }
-System.err.println("Wonky job status COMPLETED "+jobNumber);
             } else {
                 throw new JobException(exitCode);
             }
@@ -371,7 +369,7 @@
     boolean failDelay(String name) {
         double failRate = getWonkyParam(name+"fail");
         double delayTime = getWonkyParam(name+"delay");
-        System.out.println("Fail probability is: "+failRate);
+        System.out.print("State "+name+" with probability of failure "+failRate+": ");
         if(Math.random()<failRate) {
             System.out.println("Fail.");
             return false;
@@ -393,7 +391,7 @@
 
             if(op.startsWith(name+"=")) {
                 String value = op.substring(name.length()+1);
-                System.out.println("Parameter "+name+" has value "+value);
+                logger.debug("Parameter "+name+" has value "+value);
                 return Double.parseDouble(value);
             }
         }



From noreply at svn.ci.uchicago.edu  Wed Jul 30 08:46:33 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 30 Jul 2008 08:46:33 -0500 (CDT)
Subject: [Swift-commit] r2160 - trunk/docs
Message-ID: <20080730134633.88C591360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-30 08:46:33 -0500 (Wed, 30 Jul 2008)
New Revision: 2160

Modified:
   trunk/docs/README
Log:
more docbook README

Modified: trunk/docs/README
===================================================================
--- trunk/docs/README	2008-07-29 10:30:40 UTC (rev 2159)
+++ trunk/docs/README	2008-07-30 13:46:33 UTC (rev 2160)
@@ -19,8 +19,17 @@
 to place the docbook formatting stylesheets under the formatting/docbook/
 directory. This can be done with a symlink if docbook is installed elsewhere.
 
-For example, on benc's os x machine:
+For example:
 
+A) On the CI network, /home/hategan/docbook contains a docbook installation that
+can be linked like this:
+
+$ cd formatting
+$ ln -s /home/hategan/docbook/ docbook
+
+
+B) on benc's os x machine:
+
 # install docbook from DarwinPorts
 $ sudo port install docbook-xsl
 
@@ -29,10 +38,12 @@
 $ ln -s /opt/local/share/xsl/docbook-xsl/ docbook
 
 
-The buildguides.sh script will build all guides. Run it with no parameters,
-like this:
+Once the links are set up, the buildguides.sh script will build all guides
+as php.  Run it with no parameters, like this:
 
 $ ./buildguides.sh
 
-This will produced a .php file for each document. These files are
-sufficiently HTML-like that they can be opened in a local web browser.
+or use make to get HTML documents like this:
+
+$ make userguide.html
+



From noreply at svn.ci.uchicago.edu  Wed Jul 30 10:54:28 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 30 Jul 2008 10:54:28 -0500 (CDT)
Subject: [Swift-commit] r2161 - in trunk/tests/language: not-working working
	working-base
Message-ID: <20080730155428.863D21360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-30 10:54:27 -0500 (Wed, 30 Jul 2008)
New Revision: 2161

Added:
   trunk/tests/language/not-working/dataset-type-not-defined.swift
   trunk/tests/language/not-working/proc_arg_type_not_defined.swift
   trunk/tests/language/not-working/var_type_not_defined.swift
Modified:
   trunk/tests/language/working-base/013-foreach-in-proc.xml
   trunk/tests/language/working-base/014-mapper.xml
   trunk/tests/language/working-base/015-mapper-literal.xml
   trunk/tests/language/working-base/016-mapper-noparam.xml
   trunk/tests/language/working-base/022-app-procedure.xml
   trunk/tests/language/working-base/036-typedef-empty-semi.xml
   trunk/tests/language/working/013-foreach-in-proc.swift
   trunk/tests/language/working/014-mapper.swift
   trunk/tests/language/working/015-mapper-literal.swift
   trunk/tests/language/working/016-mapper-noparam.swift
   trunk/tests/language/working/022-app-procedure.swift
   trunk/tests/language/working/036-typedef-empty-semi.swift
Log:
More checks that types are defined when used.

This commit contributed by Milena Nikolic <nikolicmilena at gmail.com>


Added: trunk/tests/language/not-working/dataset-type-not-defined.swift
===================================================================
--- trunk/tests/language/not-working/dataset-type-not-defined.swift	                        (rev 0)
+++ trunk/tests/language/not-working/dataset-type-not-defined.swift	2008-07-30 15:54:27 UTC (rev 2161)
@@ -0,0 +1 @@
+myfile v<simple_mapper;pattern="*">;
\ No newline at end of file

Added: trunk/tests/language/not-working/proc_arg_type_not_defined.swift
===================================================================
--- trunk/tests/language/not-working/proc_arg_type_not_defined.swift	                        (rev 0)
+++ trunk/tests/language/not-working/proc_arg_type_not_defined.swift	2008-07-30 15:54:27 UTC (rev 2161)
@@ -0,0 +1,2 @@
+foo (file f) {
+}
\ No newline at end of file

Added: trunk/tests/language/not-working/var_type_not_defined.swift
===================================================================
--- trunk/tests/language/not-working/var_type_not_defined.swift	                        (rev 0)
+++ trunk/tests/language/not-working/var_type_not_defined.swift	2008-07-30 15:54:27 UTC (rev 2161)
@@ -0,0 +1 @@
+file f;
\ No newline at end of file

Modified: trunk/tests/language/working/013-foreach-in-proc.swift
===================================================================
--- trunk/tests/language/working/013-foreach-in-proc.swift	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working/013-foreach-in-proc.swift	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1,3 +1,4 @@
+type file;
 
 (file f) echo (string s) {
   app {

Modified: trunk/tests/language/working/014-mapper.swift
===================================================================
--- trunk/tests/language/working/014-mapper.swift	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working/014-mapper.swift	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1 +1,2 @@
+type myfile;
 myfile v<simple_mapper;pattern="*">;

Modified: trunk/tests/language/working/015-mapper-literal.swift
===================================================================
--- trunk/tests/language/working/015-mapper-literal.swift	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working/015-mapper-literal.swift	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1 +1,2 @@
+type myfile;
 myfile v<"literalfilename.txt">;

Modified: trunk/tests/language/working/016-mapper-noparam.swift
===================================================================
--- trunk/tests/language/working/016-mapper-noparam.swift	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working/016-mapper-noparam.swift	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1 +1,2 @@
+type myfile;
 myfile v<simple_mapper;>;

Modified: trunk/tests/language/working/022-app-procedure.swift
===================================================================
--- trunk/tests/language/working/022-app-procedure.swift	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working/022-app-procedure.swift	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1,3 +1,5 @@
+type file;
+
 (file f) generate (float p1) {
     app {
                 generate "-aTOP -T4" "-p" p1 "-o" @f;

Modified: trunk/tests/language/working/036-typedef-empty-semi.swift
===================================================================
--- trunk/tests/language/working/036-typedef-empty-semi.swift	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working/036-typedef-empty-semi.swift	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1,2 +1,3 @@
+type bar;
 type foo bar;
 

Modified: trunk/tests/language/working-base/013-foreach-in-proc.xml
===================================================================
--- trunk/tests/language/working-base/013-foreach-in-proc.xml	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working-base/013-foreach-in-proc.xml	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1,6 +1,14 @@
 <program xmlns="http://ci.uchicago.edu/swift/2007/07/swiftscript"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:xs="http://www.w3.org/2001/XMLSchema">
+  
+  <types>
+     	<type>
+     		<typename>file</typename>
+     		<typealias>string</typealias>
+     		<typestructure></typestructure>
+     	</type>   
+  </types>  
   <procedure name="echo">
 
       <output 
@@ -29,9 +37,9 @@
     name="sa" type="string[]"
 
      xsi:nil="true" />
-    <foreach var="s"  indexVar="i" src="line 10">
+    <foreach var="s"  indexVar="i" src="line 11">
     <in><variableReference>sa</variableReference></in>
-    <body><call proc="echo" src="line 11">
+    <body><call proc="echo" src="line 12">
         <output><arraySubscript>
          <variableReference>fa</variableReference>
          <variableReference>i</variableReference>

Modified: trunk/tests/language/working-base/014-mapper.xml
===================================================================
--- trunk/tests/language/working-base/014-mapper.xml	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working-base/014-mapper.xml	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1,6 +1,14 @@
 <program xmlns="http://ci.uchicago.edu/swift/2007/07/swiftscript"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:xs="http://www.w3.org/2001/XMLSchema">
+  
+  <types>
+     	<type>
+     		<typename>myfile</typename>
+     		<typealias>string</typealias>
+     		<typestructure></typestructure>
+     	</type>   
+  </types>  
   <dataset name="v" type="myfile">
     <mapping descriptor="simple_mapper">
       <param name="pattern"><stringConstant>*</stringConstant></param>

Modified: trunk/tests/language/working-base/015-mapper-literal.xml
===================================================================
--- trunk/tests/language/working-base/015-mapper-literal.xml	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working-base/015-mapper-literal.xml	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1,6 +1,14 @@
 <program xmlns="http://ci.uchicago.edu/swift/2007/07/swiftscript"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:xs="http://www.w3.org/2001/XMLSchema">
+  
+  <types>
+     	<type>
+     		<typename>myfile</typename>
+     		<typealias>string</typealias>
+     		<typestructure></typestructure>
+     	</type>   
+  </types>  
   <dataset name="v" type="myfile">
     <file name="literalfilename.txt"/>
   </dataset>

Modified: trunk/tests/language/working-base/016-mapper-noparam.xml
===================================================================
--- trunk/tests/language/working-base/016-mapper-noparam.xml	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working-base/016-mapper-noparam.xml	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1,6 +1,14 @@
 <program xmlns="http://ci.uchicago.edu/swift/2007/07/swiftscript"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:xs="http://www.w3.org/2001/XMLSchema">
+  
+  <types>
+     	<type>
+     		<typename>myfile</typename>
+     		<typealias>string</typealias>
+     		<typestructure></typestructure>
+     	</type>   
+  </types>  
   <dataset name="v" type="myfile">
     <mapping descriptor="simple_mapper"/>
   </dataset>

Modified: trunk/tests/language/working-base/022-app-procedure.xml
===================================================================
--- trunk/tests/language/working-base/022-app-procedure.xml	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working-base/022-app-procedure.xml	2008-07-30 15:54:27 UTC (rev 2161)
@@ -1,6 +1,14 @@
 <program xmlns="http://ci.uchicago.edu/swift/2007/07/swiftscript"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:xs="http://www.w3.org/2001/XMLSchema">
+  
+  <types>
+     	<type>
+     		<typename>file</typename>
+     		<typealias>string</typealias>
+     		<typestructure></typestructure>
+     	</type>   
+  </types>  
   <procedure name="generate">
 
       <output 

Modified: trunk/tests/language/working-base/036-typedef-empty-semi.xml
===================================================================
--- trunk/tests/language/working-base/036-typedef-empty-semi.xml	2008-07-30 13:46:33 UTC (rev 2160)
+++ trunk/tests/language/working-base/036-typedef-empty-semi.xml	2008-07-30 15:54:27 UTC (rev 2161)
@@ -3,6 +3,11 @@
          xmlns:xs="http://www.w3.org/2001/XMLSchema">
   
   <types>
+     	<type>
+     		<typename>bar</typename>
+     		<typealias>string</typealias>
+     		<typestructure></typestructure>
+     	</type>
         	<type>
      		<typename>foo</typename>
      		<typealias>bar</typealias>



From noreply at svn.ci.uchicago.edu  Wed Jul 30 15:40:33 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 30 Jul 2008 15:40:33 -0500 (CDT)
Subject: [Swift-commit] r2162 - SwiftApps/SIDGrid/config
Message-ID: <20080730204033.9956B1360015@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-30 15:40:32 -0500 (Wed, 30 Jul 2008)
New Revision: 2162

Modified:
   SwiftApps/SIDGrid/config/tc.data
Log:
added host_types to mediator on ucanl64 bcs swift was ignoring it in the sites file

Modified: SwiftApps/SIDGrid/config/tc.data
===================================================================
--- SwiftApps/SIDGrid/config/tc.data	2008-07-30 15:54:27 UTC (rev 2161)
+++ SwiftApps/SIDGrid/config/tc.data	2008-07-30 20:40:32 UTC (rev 2162)
@@ -865,7 +865,7 @@
 ANLUCTERAGRID64   whirlgif   /soft/community/SIDGrid/usr/bin/whirlgif   INSTALLED INTEL64::LINUX GLOBUS::maxwalltime=240;env::PATHPREFIX=/soft/community/SIDGrid/usr/bin,LD_LIBRARY_PATH=/soft/community/SIDGrid/usr/lib
 ANLUCTERAGRID64   Xphace   /soft/community/SIDGrid/usr/bin/Xphace   INSTALLED INTEL64::LINUX GLOBUS::maxwalltime=240;env::PATHPREFIX=/soft/community/SIDGrid/usr/bin,LD_LIBRARY_PATH=/soft/community/SIDGrid/usr/lib
 #-------------------------------------------------------
-ANLUCTERAGRID64   Mediator /soft/community/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX  GLOBUS::maxwalltime=300
+ANLUCTERAGRID64   Mediator /soft/community/SIDGrid/scripts/Mediator.py  INSTALLED INTEL64::LINUX  GLOBUS::maxwalltime=300,host_types=ia64-compute
 ANLUCTERAGRID64   echo /bin/echo  INSTALLED INTEL64::LINUX  null
 ANLUCTERAGRID64   RInvoke /soft/community/SIDGrid/scripts/RInvoke.sh  INSTALLED INTEL64::LINUX  null
 ANLUCTERAGRID64   env /bin/env INSTALLED INTEL64::LINUX  null



From noreply at svn.ci.uchicago.edu  Wed Jul 30 15:43:19 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 30 Jul 2008 15:43:19 -0500 (CDT)
Subject: [Swift-commit] r2163 - SwiftApps/SIDGrid/config
Message-ID: <20080730204319.5BD9C1360015@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-30 15:43:18 -0500 (Wed, 30 Jul 2008)
New Revision: 2163

Modified:
   SwiftApps/SIDGrid/config/sites_ranger.xml
Log:
deleted unused specs

Modified: SwiftApps/SIDGrid/config/sites_ranger.xml
===================================================================
--- SwiftApps/SIDGrid/config/sites_ranger.xml	2008-07-30 20:40:32 UTC (rev 2162)
+++ SwiftApps/SIDGrid/config/sites_ranger.xml	2008-07-30 20:43:18 UTC (rev 2163)
@@ -7,8 +7,10 @@
 </set>
   <!-- RANGER @ tg-login.ranger.tacc.teragrid.org -->
   <pool handle="RANGER" sysinfo="INTEL64::LINUX">
-    <gridftp url="gsiftp://gridftp.ranger.tacc.teragrid.org" storage="/work/00043/tg457040/sidgrid_out/{username}" major="2" minor="4" patch="3"/>
-    <jobmanager universe="vanilla" url="tg-login.ranger.tacc.teragrid.org/jobmanager-sge" major="2" minor="4" patch="3"/>
-    <workdirectory>sidgrid_out/{username}</workdirectory>
+    <profile namespace="karajan" key="initialScore">1</profile>
+    <profile namespace="karajan" key="jobThrottle">3</profile>
+    <gridftp url="gsiftp://gridftp.ranger.tacc.teragrid.org"/>
+    <execution provider="gt2" url="gatekeeper.ranger.tacc.teragrid.org" jobManager="SGE"/>
+    <workdirectory>/work/00926/tg459516/sidgrid_out/{username}</workdirectory>
   </pool>
 </config>



From noreply at svn.ci.uchicago.edu  Wed Jul 30 15:44:41 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Wed, 30 Jul 2008 15:44:41 -0500 (CDT)
Subject: [Swift-commit] r2164 - SwiftApps/SIDGrid/config
Message-ID: <20080730204441.32DCE1360015@www.ci.uchicago.edu>

Author: skenny
Date: 2008-07-30 15:44:40 -0500 (Wed, 30 Jul 2008)
New Revision: 2164

Modified:
   SwiftApps/SIDGrid/config/sites_ncsa.xml
Log:
making gt2 the default provider for ncsa, jobs fail on gt4

Modified: SwiftApps/SIDGrid/config/sites_ncsa.xml
===================================================================
--- SwiftApps/SIDGrid/config/sites_ncsa.xml	2008-07-30 20:43:18 UTC (rev 2163)
+++ SwiftApps/SIDGrid/config/sites_ncsa.xml	2008-07-30 20:44:40 UTC (rev 2164)
@@ -7,10 +7,10 @@
 </set>
   <!-- NCSAMERCURY @ grid-hg.ncsa.teragrid.org -->
   <pool handle="NCSAMERCURY">
-    <profile namespace="karajan" key="initialScore">7</profile>
-    <profile namespace="karajan" key="jobThrottle">4</profile>
+    <profile namespace="karajan" key="initialScore">1</profile>
+    <profile namespace="karajan" key="jobThrottle">2</profile>
     <gridftp url="gsiftp://gridftp-hg.ncsa.teragrid.org"/>
-    <execution provider="gt4" url="grid-hg.ncsa.teragrid.org" jobManager="PBS"/>
+    <execution provider="gt2" url="grid-hg.ncsa.teragrid.org" jobManager="PBS"/>
     <workdirectory>/usr/projects/tg-community/SIDGrid/sidgrid_out/{username}</workdirectory>
   </pool>
 </config>



From noreply at svn.ci.uchicago.edu  Thu Jul 31 08:51:31 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 31 Jul 2008 08:51:31 -0500 (CDT)
Subject: [Swift-commit] r2165 - trunk/src/org/griphyn/vdl/engine
Message-ID: <20080731135131.9DDD61360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-31 08:51:30 -0500 (Thu, 31 Jul 2008)
New Revision: 2165

Modified:
   trunk/src/org/griphyn/vdl/engine/Karajan.java
Log:
stuff that was missed from 2161 (benc's fault). This commit contributed by Milena Nikolic <nikolicmilena at gmail.com>

Modified: trunk/src/org/griphyn/vdl/engine/Karajan.java
===================================================================
--- trunk/src/org/griphyn/vdl/engine/Karajan.java	2008-07-30 20:44:40 UTC (rev 2164)
+++ trunk/src/org/griphyn/vdl/engine/Karajan.java	2008-07-31 13:51:30 UTC (rev 2165)
@@ -176,8 +176,10 @@
 											
 				StringTemplate st = template("typeDef");
 				st.setAttribute("name", typeName);
-				if (typeAlias != null && !typeAlias.equals("") && !typeAlias.equals("string"))
-					 st.setAttribute("type", typeAlias);
+				if (typeAlias != null && !typeAlias.equals("") && !typeAlias.equals("string")) {
+					checkIsTypeDefined(typeAlias);
+					st.setAttribute("type", typeAlias);
+				}
 										
 				TypeStructure ts = theType.getTypestructure();
 				for (int j = 0; j < ts.sizeOfMemberArray(); j++) {				
@@ -228,6 +230,7 @@
 				procST.setAttribute("optargs", paramST);
 			else
 				procST.setAttribute("arguments", paramST);
+			checkIsTypeDefined(param.getType().getLocalPart());
 			innerScope.addVariable(param.getName(), param.getType().getLocalPart());
 		}
 		for (int i = 0; i < proc.sizeOfInputArray(); i++) {
@@ -238,6 +241,7 @@
 				procST.setAttribute("optargs", paramST);
 			else
 				procST.setAttribute("arguments", paramST);
+			checkIsTypeDefined(param.getType().getLocalPart());
 			outerScope.addVariable(param.getName(), param.getType().getLocalPart());
 		}
 
@@ -272,23 +276,7 @@
 		variableST.setAttribute("type", var.getType().getLocalPart());
 		variableST.setAttribute("isArray", Boolean.valueOf(var.getIsArray1()));
 
-		/* Checking if variable type is defined */
-		String type = var.getType().getLocalPart();
-		while (type.substring(type.length() - 2).equals("[]"))
-			type = type.substring(0, type.length() - 2);		
-		if (!type.equals("int") && !type.equals("float") && !type.equals("string") 
-				&& !type.equals("boolean") && !type.equals("external")) {
-			boolean typeDefined = false;
-			if (types != null) {
-				for (int i = 0; i < types.sizeOfTypeArray(); i++)
-					if (types.getTypeArray(i).getTypename().equals(type)) {
-						typeDefined = true;
-						break;
-					}
-			}
-			if (!typeDefined)
-				throw new CompilationException("Type " + type + " is not defined.");
-		}
+		checkIsTypeDefined(var.getType().getLocalPart());
 		
 		if(!var.isNil()) {
 			StringTemplate exprST = expressionToKarajan(var.getAbstractExpression(),scope);
@@ -319,6 +307,9 @@
 		if (dataset.isSetIsArray1()) {
 			datasetST.setAttribute("isArray", Boolean.valueOf(dataset.getIsArray1()));
 		}
+		
+		checkIsTypeDefined(dataset.getType().getLocalPart());
+		
 		if (dataset.getFile() != null) {
 			StringTemplate fileST = new StringTemplate("file");
 			fileST.setAttribute("name", escapeQuotes(dataset.getFile().getName()));
@@ -341,6 +332,24 @@
 		}
 		scope.addVariable(dataset.getName(), dataset.getType().getLocalPart());
 	}
+	
+	void checkIsTypeDefined(String type) throws CompilationException {		
+		while (type.substring(type.length() - 2).equals("[]"))
+			type = type.substring(0, type.length() - 2);		
+		if (!type.equals("int") && !type.equals("float") && !type.equals("string") 
+				&& !type.equals("boolean") && !type.equals("external")) {
+			boolean typeDefined = false;
+			if (types != null) {
+				for (int i = 0; i < types.sizeOfTypeArray(); i++)
+					if (types.getTypeArray(i).getTypename().equals(type)) {
+						typeDefined = true;
+						break;
+					}
+			}
+			if (!typeDefined)
+				throw new CompilationException("Type " + type + " is not defined.");
+		}
+	}
 
 	public void assign(Assign assign, VariableScope scope) throws CompilationException {
 		try {



From noreply at svn.ci.uchicago.edu  Thu Jul 31 10:34:35 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 31 Jul 2008 10:34:35 -0500 (CDT)
Subject: [Swift-commit] r2166 - trunk/docs
Message-ID: <20080731153435.938391360015@www.ci.uchicago.edu>

Author: benc
Date: 2008-07-31 10:34:35 -0500 (Thu, 31 Jul 2008)
New Revision: 2166

Modified:
   trunk/docs/userguide.xml
Log:
docs on recently added profile entries

Modified: trunk/docs/userguide.xml
===================================================================
--- trunk/docs/userguide.xml	2008-07-31 13:51:30 UTC (rev 2165)
+++ trunk/docs/userguide.xml	2008-07-31 15:34:35 UTC (rev 2166)
@@ -2248,6 +2248,9 @@
 allows the initial score for rate limiting and site selection to be set to
 a value other than 0.
 			</para>
+			<para id="profile.karajan.delayBase">delayBase - controls how much a site will be delayed when it performs poorly. With each reduction
+in a sites score by 1, the delay between execution attempts will increase by
+a factor of delayBase.</para>
 		</section>
 		<section id="profile.swift"><title>swift namespace</title>
 			<para id="profile.swift.storagesize">storagesize limits the
@@ -2282,6 +2285,10 @@
 			<para id="profile.globus.condor_requirements"><property>condor_requirements</property> allows a requirements string to be specified
 when Condor is used as an LRM behind GRAM2. Example: &lt;profile namespace="globus" key="condor_requirements">Arch == &quot;X86_64&quot; || Arch=&quot;INTEL&quot;&lt;/profile>
 			</para>
+			<para id="profile.globus.coastersPerNode">Specifies
+the number of coaster workers to be run on each node. This profile entry
+is used by the coaster provider.
+			</para>
 		</section>
 		<section id="profile.env"><title>env namespace</title>
 			<para>
@@ -2531,7 +2538,18 @@
 www11.i2u2.org.passphrase=XXXX
 </screen>
 		</para>
-
+		<para>
+For execution using the CoG Coaster mechanism, the coaster provider
+should be used:
+<screen>
+&lt;execution provider="coaster" url="tg-grid.uc.teragrid.org"
+    jobmanager="gt2:gt2:pbs" />
+</screen>
+with the jobmanager parameter specifying: the cog provider to use to
+submit the coaster head job; the cog provider to use to submit
+coaster worker jobs; and optionally the jobmanager to be used by
+worker submission.
+		</para>
 </section>
 <section><title>Other site catalog parameters</title>
 



From noreply at svn.ci.uchicago.edu  Thu Jul 31 17:38:28 2008
From: noreply at svn.ci.uchicago.edu (noreply at svn.ci.uchicago.edu)
Date: Thu, 31 Jul 2008 17:38:28 -0500 (CDT)
Subject: [Swift-commit] r2167 -
	provider-deef/src/org/globus/cog/abstraction/impl/execution/deef
Message-ID: <20080731223828.A91271360015@www.ci.uchicago.edu>

Author: wilde
Date: 2008-07-31 17:38:27 -0500 (Thu, 31 Jul 2008)
New Revision: 2167

Modified:
   provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java
Log:
Remove old commented-out methods from prior testing

Modified: provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java
===================================================================
--- provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java	2008-07-31 15:34:35 UTC (rev 2166)
+++ provider-deef/src/org/globus/cog/abstraction/impl/execution/deef/ResourcePool.java	2008-07-31 22:38:27 UTC (rev 2167)
@@ -218,45 +218,6 @@
 	*/
     }
 
-    public static String getMachNamePort__ORIG__(Notification userNot){
-        String machName = "";
-        //String machIP;
-        try {
-	    machName = java.net.InetAddress.getLocalHost().getCanonicalHostName();
-	    machName = machName +  ":" + userNot.recvPort;
-	    //machIP = java.net.InetAddress.getLocalHost().getHostAddress();
-	    logger.debug("WORKER: Machine ID = " + machName);
-
-        } catch (Exception e) {
-            logger.error("WORKER: ERROR: java.net.InetAddress.getLocalHost().getHostName() failed " + e);
-        }
-	return machName;
-    }
-
-/**** From Ioan:
-   public String getMachNamePort__IOANS_EMAIL__(int recvPort)
-   {
-       String machName = new String("localhost:"+recvPort);
-       try
-       {
-           String method = new String("overide");
-           if (NOTIFICATION_ENDPOINT == null)
-           {
-               NOTIFICATION_ENDPOINT = java.net.InetAddress.getLocalHost().getCanonicalHostName();
-               method = new String("automatic");
-           }
-           machName = NOTIFICATION_ENDPOINT +  ":" + recvPort;
-           System.out.println("Notification Endpoint (" + method + "): " + NOTIFICATION_ENDPOINT);
-       }
-       catch (Exception e)
-       {
-           System.out.println("WORKER: ERROR: java.net.InetAddress.getLocalHost().getHostName() failed " + e);
-           if (DEBUG) e.printStackTrace();
-       }
-       return machName;
-   }
-****/
-
     public static String getMachNamePort(Notification userNot){
         //String machIP = VDL2Config.getIP();
 	String machIP = CoGProperties.getDefault().getIPAddress();