[Swift-commit] r6641 - SwiftApps/swiftrun-demo

[davidk at ci.uchicago.edu]

Author: davidk
Date: 2013-07-12 11:59:16 -0500 (Fri, 12 Jul 2013)
New Revision: 6641

Added:
   SwiftApps/swiftrun-demo/README
   SwiftApps/swiftrun-demo/catsn.swift
   SwiftApps/swiftrun-demo/data.txt
   SwiftApps/swiftrun-demo/swift.properties
Log:
Example scripts


Added: SwiftApps/swiftrun-demo/README
===================================================================
--- SwiftApps/swiftrun-demo/README	                        (rev 0)
+++ SwiftApps/swiftrun-demo/README	2013-07-12 16:59:16 UTC (rev 6641)
@@ -0,0 +1,77 @@
+* You only need only one configuration file: swift.properties. 
+
+* Configuration becomes easier. No XML.
+
+* swift.properties file contains information about sites, 
+  apps, and properties.
+
+* Provides a mechanism for site templates
+
+* Gives us an easy way to simplify configuration names
+
+* Goal to be backward compatible with old swift syntax.
+
+* Run directories are created for each run, named runNNNN and incremented
+  with every call.
+
+* Log files and *.d directories go to the runNNNN directory.
+
+
+How to Run
+----------
+This has only been tested on Midway at the moment. There are two sites 
+defined in swift.properties, westmere and sandyb. Check sinfo to see 
+which queues have available nodes to avoid getting stuck in the queue.
+
+All the information we need for this run is stored in swift.properties.
+
+To run:
+-----
+nswift -site westmere catsn.swift
+nswift -site sandyb catsn.swift
+nswift -site westmere,sandyb catsn.swift
+-----
+
+How it works
+------------
+All configuration information is stored in swift.properties.
+
+cf values
+~~~~~~~~~
+Swift.properties will contain all of your swift configuration values. 
+For example:
+
+-----
+sitedir.keep=true
+-----
+
+Site definitions
+~~~~~~~~~~~~~~~~
+There is no need to use a sites.xml file, because swift.properties can contain
+full site definitions. Here is an example defining the "sandyb" site:
+
+-----
+site.sandyb.provider=local:slurm
+site.sandyb.jobspernode=16
+site.sandyb.maxWalltime=00:05:00
+site.sandyb.queue=sandyb
+site.sandyb.initialScore=10000
+site.sandyb.filesystem=local
+site.sandyb.workdirectory=/scratch/midway/$USER/work
+-----
+
+App definitions
+~~~~~~~~~~~~~~~
+There is no need to have a tc.data file. Apps can be defined in swift.properties.
+
+Example:
+-----
+app.sandyb.echo=/bin/echo
+-----
+
+You can use a wildcard to make an app available for every site that you have defined.
+Example:
+-----
+app.*.echo=/bin/echo
+-----
+

Added: SwiftApps/swiftrun-demo/catsn.swift
===================================================================
--- SwiftApps/swiftrun-demo/catsn.swift	                        (rev 0)
+++ SwiftApps/swiftrun-demo/catsn.swift	2013-07-12 16:59:16 UTC (rev 6641)
@@ -0,0 +1,13 @@
+type file;
+
+app (file o) cat (file i)
+{
+  cat @i stdout=@o;
+}
+
+file out[]<simple_mapper; location="output", prefix="catsn.",suffix=".out">;
+
+foreach j in [1:toInt(arg("n","1"))] {
+  file data <"data.txt">;
+  out[j] = cat(data);
+}

Added: SwiftApps/swiftrun-demo/data.txt
===================================================================
--- SwiftApps/swiftrun-demo/data.txt	                        (rev 0)
+++ SwiftApps/swiftrun-demo/data.txt	2013-07-12 16:59:16 UTC (rev 6641)
@@ -0,0 +1 @@
+fjdlksafjlsd

Added: SwiftApps/swiftrun-demo/swift.properties
===================================================================
--- SwiftApps/swiftrun-demo/swift.properties	                        (rev 0)
+++ SwiftApps/swiftrun-demo/swift.properties	2013-07-12 16:59:16 UTC (rev 6641)
@@ -0,0 +1,22 @@
+site.sandyb.provider=local:slurm
+site.sandyb.jobspernode=16
+site.sandyb.maxWalltime=00:05:00
+site.sandyb.queue=sandyb
+site.sandyb.initialScore=10000
+site.sandyb.filesystem=local
+site.sandyb.workdirectory=/scratch/midway/$USER/work
+
+site.westmere.provider=local:slurm
+site.westmere.jobsPerNode=12
+site.westmere.maxWalltime=00:05:00
+site.westmere.queue=westmere
+site.westmere.initialScore=10000
+site.westmere.filesystem=local
+site.westmere.workdirectory=/scratch/midway/$USER/work
+
+app.sandyb.echo=/bin/echo
+app.westmere.echo=/bin/echo
+app.*.cat=/bin/cat
+
+sitedir.keep=true
+wrapperlog.always.transfer=true