[Swift-commit] r7013 - in SwiftTutorials/swift-cray-tutorial: . app doc part01 part02 part03 part04 part05 part06
wilde at ci.uchicago.edu
wilde at ci.uchicago.edu
Tue Aug 27 18:40:14 CDT 2013
Author: wilde
Date: 2013-08-27 18:40:14 -0500 (Tue, 27 Aug 2013)
New Revision: 7013
Added:
SwiftTutorials/swift-cray-tutorial/part05/OLD.p5.swift
SwiftTutorials/swift-cray-tutorial/part05/OLDbias.dat
SwiftTutorials/swift-cray-tutorial/part05/multipools
SwiftTutorials/swift-cray-tutorial/part05/p5.swift
Removed:
SwiftTutorials/swift-cray-tutorial/part05/bias.dat
SwiftTutorials/swift-cray-tutorial/part05/p5.swift
Modified:
SwiftTutorials/swift-cray-tutorial/app/simulate.sh
SwiftTutorials/swift-cray-tutorial/app/stats.sh
SwiftTutorials/swift-cray-tutorial/doc/README
SwiftTutorials/swift-cray-tutorial/doc/asciidoc.css
SwiftTutorials/swift-cray-tutorial/doc/push.sh
SwiftTutorials/swift-cray-tutorial/part01/apps
SwiftTutorials/swift-cray-tutorial/part01/p1.swift
SwiftTutorials/swift-cray-tutorial/part02/p2.swift
SwiftTutorials/swift-cray-tutorial/part03/apps
SwiftTutorials/swift-cray-tutorial/part03/p3.swift
SwiftTutorials/swift-cray-tutorial/part04/apps
SwiftTutorials/swift-cray-tutorial/part04/p4.swift
SwiftTutorials/swift-cray-tutorial/part05/apps
SwiftTutorials/swift-cray-tutorial/part06/apps
SwiftTutorials/swift-cray-tutorial/part06/p6.swift
SwiftTutorials/swift-cray-tutorial/setup.sh
Log:
Revisions for Cray.
Modified: SwiftTutorials/swift-cray-tutorial/app/simulate.sh
===================================================================
--- SwiftTutorials/swift-cray-tutorial/app/simulate.sh 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/app/simulate.sh 2013-08-27 23:40:14 UTC (rev 7013)
@@ -22,7 +22,7 @@
printf "Running as user: "; /usr/bin/id
printparams
printf "\nEnvironment:\n\n"
- /bin/env | /bin/sort
+ printenv | sort
}
addsims() {
Modified: SwiftTutorials/swift-cray-tutorial/app/stats.sh
===================================================================
--- SwiftTutorials/swift-cray-tutorial/app/stats.sh 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/app/stats.sh 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,9 +1,18 @@
#! /bin/sh
+log() {
+ printf "\nCalled as: $0: $cmdargs\n\n"
+ printf "Start time: "; /bin/date
+ printf "Running on node: "; /bin/hostname
+ printf "Running as user: "; /usr/bin/id
+ printf "\nEnvironment:\n\n"
+ printenv | sort
+}
+
awk '
{ sum += $1}
END { printf("%d\n",sum/NR) }
' $*
-
+log 1>&2
Modified: SwiftTutorials/swift-cray-tutorial/doc/README
===================================================================
--- SwiftTutorials/swift-cray-tutorial/doc/README 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/doc/README 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,29 +1,31 @@
Swift Cray Tutorial
===================
-//// Comments:
+////
This is the asciidoc input file.
-The content below is also viewable as a plain-text README file.
+Its content is viewable as a plain-text README file.
////
This tutorial is viewable at:
-http://swift-lang.org/tutorials/swift-cray-tutorial.html
+http://swift-lang.org/tutorials/cray/swift-cray-tutorial.html
-//// Tutorial Outline:
+////
-Introductory exercises, local, on login node:
+Tutorial Outline:
+Introductory example, running apps locally on login node:
+
p1 - Run an application under Swift
p2 - Parallel loops with foreach
p3 - Merging/reducing the results of a parallel foreach loop
-Compute-node exercises:
+Compute-node exercises, running apps via qsub and aprun:
- p4 - Running apps on Cray comupte nodes
+ p4 - Running apps on Cray compute nodes
p5 - Running on multiple pools of compute nodes
- p6 - A more complex workflow pattern
+ p6 - Running a more complex workflow pattern
////
@@ -271,80 +273,119 @@
.p1.swift
-----
-sys::[cat -n ../part01/p1.swift]
+sys::[cat ../part01/p1.swift]
-----
-The sites.xml file included in each part directory gives Swift
-information about the machines we will be running on. It defines
+
+To run this script, run the following command:
+-----
+$ cd part01
+$ swift p1.swift
+Swift 0.94.1 RC2 swift-r6895 cog-r3765
+
+RunID: 20130827-1413-oa6fdib2
+Progress: time: Tue, 27 Aug 2013 14:13:33 -0500
+Final status: Tue, 27 Aug 2013 14:13:33 -0500 Finished successfully:1
+$ cat sim.out
+ 84
+$ swift p1.swift
+$ cat sim.out
+ 36
+-----
+
+To cleanup the directory and remove all outputs (including the log
+files and directories that Swift generates), run the cleanup script
+which is located in the tutorial PATH:
+
+-----
+$ cleanup.sh
+------
+
+NOTE: You'll also find two Swift configuration files in each `partNN`
+directory of this tutorial. These specify the environment-specific
+details of where to find application programs (file `apps`) and where
+to run them (file `sites.xml`). These files will be explained in more
+detail in parts 4-6, and can be ignored for now.
+
+////
+It defines
things like the work directory, the scheduler to use, and how to
control parallelism. The sites.xml file below will tell Swift to run
on the local machine only, and run just 1 task at a time.
.sites.xml
-----
-sys::[cat -n ../part01/sites.xml]
+sys::[cat ../part01/sites.xml]
-----
-The app file translates from a Swift app function to the path of an
-executable on the file system. In this case, it translates from
-"simulate" to simulate.sh and assumes that simulate.sh will be
-available in your $PATH.
+ In this case, it
+indicates that the app "simulate" (the first token in the command line
+declaration of the function `simulation`, at line NNN) is located in the file
+simulate.sh and (since the path `simulate.sh` is specified with no
+directory components) Swift expects that the `simulate.sh` executable
+will be available in your $PATH.
.apps
-----
-sys::[cat -n ../part01/apps]
+sys::[cat ../part01/apps]
-----
-To run this script, run the following command:
------
-$ cd part01
-$ swift p1.swift
------
+////
-The simulate application gets translated to simulate.sh within the 'apps' file.
+Part 2: Running an ensemble of many apps in parallel with a "foreach" loop
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-NOTE: Since the file you created is not named, swift will generate a random
-name for the file in a directory called _concurrent. To view the created
-output, run "cat _concurrent/*"
+The `p2.swift` script introduces the `foreach` parallel iteration
+construct to run many concurrent simulations.
-To cleanup the directory and remove all outputs, run:
------
-$ ./clean.sh
-------
-
-
-
-Part 2: Running an ensemble of many apps in parallel with "foreach" loops
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The p2.swift script introduces the foreach loop. This script runs many
-simulations. The script also shows an example of naming the files. The output files
-are now called sim_N.out.
-
image::part02.png[align="center"]
.p2.swift
-----
-sys::[cat -n ../part02/p2.swift]
+sys::[cat ../part02/p2.swift]
-----
-To run the script:
+The script also shows an
+example of naming the output files of an ensemble run. In this case, the output files will be named
+`output/sim_N.out`.
+
+To run the script and view the output:
-----
$ cd ../part02
$ swift p2.swift
+$ ls output
+sim_0.out sim_1.out sim_2.out sim_3.out sim_4.out sim_5.out sim_6.out sim_7.out sim_8.out sim_9.out
+$ more output/*
+::::::::::::::
+output/sim_0.out
+::::::::::::::
+ 44
+::::::::::::::
+output/sim_1.out
+::::::::::::::
+ 55
+...
+::::::::::::::
+output/sim_9.out
+::::::::::::::
+ 82
-----
Part 3: Analyzing results of a parallel ensemble
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-p3.swift introduces a postprocessing step. After all the parallel
-simulations have completed, the files created by simulation.sh will be
-averaged by stats.sh.
+After all the parallel simulations in an ensemble run have completed,
+its typically necessary to gather and analyze their results with some
+kind of post-processing analysis program or script. p3.swift
+introduces such a postprocessing step. In this case, the files created
+by all of the parallel runs of `simulation.sh` will be averaged by by
+the trivial "analysis application" `stats.sh`:
image::part03.png[align="center"]
.p3.swift
----
-sys::[cat -n ../part03/p3.swift]
+sys::[cat ../part03/p3.swift]
----
To run:
@@ -353,55 +394,344 @@
$ swift p3.swift
----
+Note that in `p3.swift` we expose more of the capabilities of the
+`simulate.sh` application to the `simulation()` app function:
+-----
+app (file o) simulation (int sim_steps, int sim_range, int sim_values)
+{
+ simulate "--timesteps" sim_steps "--range" sim_range "--nvalues" sim_values stdout=@filename(o);
+}
+-----
+
+`p3.swift` also shows how to fetch application-specific values from
+the `swift` command line in a Swift script using `@arg()` which
+accepts a keyword-style argument and its default value:
+
+-----
+int nsim = @toInt(@arg("nsim","10"));
+int steps = @toInt(@arg("steps","1"));
+int range = @toInt(@arg("range","100"));
+int values = @toInt(@arg("values","5"));
+-----
+
+Now we can specify that more runs should be performed and that each should run for more timesteps, and produce more that one value each, within a specified range, using command line arguments placed after the Swift script name in the form `-parameterName=value`:
+
+-----
+$ swift p3.swift -nsim=3 -steps=10 -values=4 -range=1000000
+
+Swift 0.94.1 RC2 swift-r6895 cog-r3765
+
+RunID: 20130827-1439-s3vvo809
+Progress: time: Tue, 27 Aug 2013 14:39:42 -0500
+Progress: time: Tue, 27 Aug 2013 14:39:53 -0500 Active:2 Stage out:1
+Final status: Tue, 27 Aug 2013 14:39:53 -0500 Finished successfully:4
+
+$ ls output/
+average.out sim_0.out sim_1.out sim_2.out
+$ more output/*
+::::::::::::::
+output/average.out
+::::::::::::::
+651368
+::::::::::::::
+output/sim_0.out
+::::::::::::::
+ 735700
+ 886206
+ 997391
+ 982970
+::::::::::::::
+output/sim_1.out
+::::::::::::::
+ 260071
+ 264195
+ 869198
+ 933537
+::::::::::::::
+output/sim_2.out
+::::::::::::::
+ 201806
+ 213540
+ 527576
+ 944233
+-----
+
+Now try running (`-nsim=`) 100 simulations of (`-steps=`) 1 second each:
+
+-----
+$ swift p3.swift -nsim=100 -steps=1
+Swift 0.94.1 RC2 swift-r6895 cog-r3765
+
+RunID: 20130827-1444-rq809ts6
+Progress: time: Tue, 27 Aug 2013 14:44:55 -0500
+Progress: time: Tue, 27 Aug 2013 14:44:56 -0500 Selecting site:79 Active:20 Stage out:1
+Progress: time: Tue, 27 Aug 2013 14:44:58 -0500 Selecting site:58 Active:20 Stage out:1 Finished successfully:21
+Progress: time: Tue, 27 Aug 2013 14:44:59 -0500 Selecting site:37 Active:20 Stage out:1 Finished successfully:42
+Progress: time: Tue, 27 Aug 2013 14:45:00 -0500 Selecting site:16 Active:20 Stage out:1 Finished successfully:63
+Progress: time: Tue, 27 Aug 2013 14:45:02 -0500 Active:15 Stage out:1 Finished successfully:84
+Progress: time: Tue, 27 Aug 2013 14:45:03 -0500 Finished successfully:101
+Final status: Tue, 27 Aug 2013 14:45:03 -0500 Finished successfully:101
+-----
+
+We can see from Swift's "progress" status that the tutorial's default
+`sites.xml` parameters for local execution allow Swift to run up to 20
+application invocations concurrently on the login node. We'll look at
+this in more detail in the next sections where we execute applications
+on the site's compute nodes.
+
+
Running applications on Cray compute nodes with Swift
-----------------------------------------------------
Part 4: Running a parallel ensemble on Cray compute nodes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-p4.swift is the first script that will run our mock "simulation" jobs
-on Cray compute nodes. The script is the same as `p3.swift`.
+p4.swift will run our mock "simulation"
+applications on Cray compute nodes. The script is similar to as
+`p3.swift`, but specifies that each simulation app invocation should
+additionally return the log file which the application writes to
+`stderr`, making the parallel portion of the script behave like this:
image::part04.png[align="center"]
.p4.swift
----
-sys::[cat -n ../part04/p4.swift]
+sys::[cat ../part04/p4.swift]
----
-To run:
-----
-$ swift p4.swift
-----
+Now when you run `swift p4.swift` you'll see that two types output
+files will placed in the `output/` directory: `sim_N.out` and
+`sim_N.log`. The log files provide data on the runtime environment of
+each app invocation. For example:
-Output files will be named output/sim_N.out.
+-----
+$ cat output/sim_0.log
+Called as: /home/users/p01532/swift-cray-tutorial/app/simulate.sh: --timesteps 1 --range 100 --nvalues 5
-In order for our .swift scripts to run apps on compute nodes, we
-specify a more complex sites.xml file. The one in `part04` is shown
-below.
+Start time: Tue Aug 27 12:17:43 CDT 2013
+Running on node: nid00018
+Running as user: uid=61532(p01532) gid=61532 groups=61532
-.sites.xml
+Simulation parameters:
+
+bias=0
+biasfile=none
+initseed=none
+log=yes
+paramfile=none
+range=100
+scale=1
+seedfile=none
+timesteps=1
+output width=8
+
+Environment:
+
+ALPS_APP_DEPTH=32
+ASSEMBLER_X86_64=/opt/cray/cce/8.2.0.173/cray-binutils/x86_64-unknown-linux-gnu/bin/as
+ASYNCPE_DIR=/opt/cray/xt-asyncpe/5.23.02
+ASYNCPE_VERSION=5.23.02
+...
-----
-sys::[cat -n ../part04/sites.xml]
+
+To tell Swift to run the apps on compute nodes, we specify in the
+`apps` file that the apps should be executed on the `raven` site
+(instead of the `localhost` site). We can specify the location of
+each app in the third field of the `apps` file, with either an
+absolute pathname or the name of an executable to be located in
+`PATH`). Here we use the latter form:
+
-----
+$ cat apps
+raven simulate simulate.sh
+raven stats stats.sh
+-----
-Below is the apps file used in this part.
+You can experiment, for example, with an alternate version of stats.sh by specfying that app's location explicitly:
-Since we are running on a single local cluster, and have one or more
-filesystems that are identically mounted on the login node where we
-are running the `swift` command and on the compute nodes where the
-apps will execute, we can specify the location of each app by a a line
-in the `apps` file, with a pathname (absolute, or a name to be located
-in PATH):
+-----
+$ cat apps
+raven simulate simulate.sh
+raven stats /home/users/p01532/bin/my-alt-stats.sh
+-----
-.apps
+We can see that when we run many apps requesting a larger set of nodes (6), we are indeed running on the compute nodes:
-----
-sys::[cat -n ../part04/apps]
+$ swift p4.swift -nsim=1000 -steps=1
+Swift 0.94.1 RC2 swift-r6895 cog-r3765
+
+RunID: 20130827-1638-t23ax37a
+Progress: time: Tue, 27 Aug 2013 16:38:11 -0500
+Progress: time: Tue, 27 Aug 2013 16:38:12 -0500 Initializing:966
+Progress: time: Tue, 27 Aug 2013 16:38:13 -0500 Selecting site:499 Submitting:500 Submitted:1
+Progress: time: Tue, 27 Aug 2013 16:38:14 -0500 Selecting site:499 Stage in:1 Submitted:500
+Progress: time: Tue, 27 Aug 2013 16:38:16 -0500 Selecting site:499 Submitted:405 Active:95 Stage out:1
+Progress: time: Tue, 27 Aug 2013 16:38:17 -0500 Selecting site:430 Submitted:434 Active:66 Stage out:1 Finished successfully:69
+Progress: time: Tue, 27 Aug 2013 16:38:18 -0500 Selecting site:388 Submitted:405 Active:95 Stage out:1 Finished successfully:111
+...
+Progress: time: Tue, 27 Aug 2013 16:38:30 -0500 Stage in:1 Submitted:93 Active:94 Finished successfully:812
+Progress: time: Tue, 27 Aug 2013 16:38:31 -0500 Submitted:55 Active:95 Stage out:1 Finished successfully:849
+Progress: time: Tue, 27 Aug 2013 16:38:32 -0500 Active:78 Stage out:1 Finished successfully:921
+Progress: time: Tue, 27 Aug 2013 16:38:34 -0500 Active:70 Stage out:1 Finished successfully:929
+Progress: time: Tue, 27 Aug 2013 16:38:37 -0500 Stage in:1 Finished successfully:1000
+Progress: time: Tue, 27 Aug 2013 16:38:38 -0500 Stage out:1 Finished successfully:1000
+Final status: Tue, 27 Aug 2013 16:38:38 -0500 Finished successfully:1001
+
+$ grep "on node:" output/*log | head
+output/sim_0.log:Running on node: nid00063
+output/sim_100.log:Running on node: nid00060
+output/sim_101.log:Running on node: nid00061
+output/sim_102.log:Running on node: nid00032
+output/sim_103.log:Running on node: nid00060
+output/sim_104.log:Running on node: nid00061
+output/sim_105.log:Running on node: nid00032
+output/sim_106.log:Running on node: nid00060
+output/sim_107.log:Running on node: nid00061
+output/sim_108.log:Running on node: nid00062
+
+$ grep "on node:" output/*log | awk '{print $4}' | sort | uniq -c
+ 158 nid00032
+ 156 nid00033
+ 171 nid00060
+ 178 nid00061
+ 166 nid00062
+ 171 nid00063
+$ hostname
+raven
+$ hostname -f
+nid00008
-----
-Larger runs
-~~~~~~~~~~~
+Swift's `sites.xml` configuration file allows many parameters to
+specify how jobs should be run on a given cluster. Consider for
+example that Raven has several queues, each with limitiations on the
+size of jobs that can be run in them. All Raven queues will only run
+2 jobs per user at one. The Raven queue "small" will only allow up to
+4 nodes per job and 1 hours of walltime per job. The following
+site.xml parameters will allow us to match this:
+
+-----
+ <profile namespace="globus" key="queue">small</profile>
+ <profile namespace="globus" key="slots">2</profile>
+ <profile namespace="globus" key="maxNodes">4</profile>
+ <profile namespace="globus" key="nodeGranularity">4</profile>
+-----
+
+To run large jobs, we can specify:
+
+-----
+ <profile namespace="globus" key="slots">2</profile>
+ <profile namespace="globus" key="maxNodes">8</profile>
+ <profile namespace="globus" key="nodeGranularity">8</profile>
+ <profile namespace="karajan" key="jobThrottle">50.0</profile>
+ <profile namespace="globus" key="maxTime">21600</profile>
+ <profile namespace="globus" key="lowOverAllocation">10000</profile>
+ <profile namespace="globus" key="highOverAllocation">10000</profile>
+-----
+
+This will enable 512 Swift apps (2 x 8 x 32) to run concurrently
+within 2 8-node jobs on Raven's 32-core nodes. It results in the
+following two PBS jobs submitted by Swift to "provision" compute nodes
+to run thousands of apps, 512 at a time:
+
+-----
+$ qstat -u $USER
+
+Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
+--------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
+288637.sdb p01532 medium B0827-2703 -- 8 256 -- 05:59 Q --
+288638.sdb p01532 medium B0827-2703 -- 8 256 -- 05:59 Q --
+-----
+
+The following section is a summary of the important `sites.xml`
+attributes for running apps on Cray systems. Many of these attributes
+can be set the same for all Swift users of a given system; only a few
+of the attributes need be overridden by users. We explain these
+attributes in detail here to show the degree of control afforded by
+Swift over application execution. Most users will use templates for a
+given Cray system, only changing a few parameters to meet any unique
+needs of their application workflows.
+
+////
+.sites.xml
+-----
+sys::[egrep -v '<.xml|<config|</config' ../part04/sites.xml | cat -n ]
+-----
+////
+
+The additional attributes in the `sites.xml` file (described here
+without their XML formatting) specify that Swift should run
+applications on Raven in the following manner:
+
+`execution provider coaster, jobmanager local:pbs` specifies that
+Swift should run apps using its "coaster" provider, which submits
+"pilot jobs" using qsub. These pilot jobs hold on to compute nodes and
+allow Swift to run many app invocations within a single job. This
+mechanism is described in
+http://www.swift-lang.org/papers/UCC-coasters.pdf[this paper from UCC-2011]
+
+`profile` tags specify additional attributes for the execution
+provider. (A "provider" is like a driver which knows how to handle
+site-specific aspects of app execution). The attributes are grouped
+into various "namespaces", but we can ignore this for now).
+
+The `env` key `PATHPREFIX` specifies that our tutorial `app` directory
+(`../app`) will be placed at the front of PATH to locate the app on
+the compute node.
+
+`queue small` specifies that pilot (coaster) jobs to run apps will be
+submitted to Raven's `small` queue.
+
+`providerAttributes pbs.aprun;pbs.mpp;depth=32` specifies some
+Cray-specific attributes: that jobs should use Cray-specific PBS "mpp"
+resource attributes (eg `mppwidth` and `mppnppn`) and an mppdepth of
+32 (because we will be running one coaster process per node, and
+Raven's XE6 dual IL-16 nodes have a depth of 32 processing elements
+(PEs).
+
+`jobsPerNode 32` tells Swift that each coaster should run up to 32
+concurrent apps. This can be reduced to place fewer apps per node, eg
+if each app needs more memory (or, rarely, greater than 32, e.g. if the apps are
+IO-bound or for benchmark experiments, etc).
+
+`slots 2` specifies that Swift will run up to 2 concurrent PBS jobs,
+and `maxNodes 1` specifies that each of these jobs will request only 1
+compute node.
+
+`maxWallTime 00:01:00` specifies that Swift should allow each app to
+run for up to one minute of walltime within the larger pilot job. In
+this example Swift will dynamically determine the total PBS walltime
+needed for the pilot job, but this can be specified manually using
+attributes `maxtime` along with `highOverAllocation` and
+`lowOverAllocation`.
+
+`jobThrottle 3.20` specifies that Swift should allow up to 320 apps to
+run on the `raven` site at once. This is typically set to a number
+greater than or equal to the number of slots x compute nodes x apps
+per node (`jobsPerNode` attribute).
+
+`initialscore 10000` is specified to override Swift's automatic
+throttling, and forces an actual throttle value of approximately
+(specifically 1 over) `jobThrottle` * 100 to be used.
+
+The last two attributes specify where and how Swift should perform
+data management. `workdirectory lus/scratch/{env.USER}/swiftwork`
+specifies where the Swift "application execution sanbox directory"
+used for each app will be located. In some situations this can be a
+directory local to the compute node (eg, for Cray systems, `/dev/shm`
+or `/tmp`, if those are writable by user jobs and the nodes have
+sufficient space in these RAM-based filesystems).
+
+Finally, `stagingMethod sfs` specifies that Swift will copy data to
+and from the shared file system to the application sandbox
+directories.
+
+////
+
+Performing larger Swift runs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
To test with larger runs, there are two changes that are required. The first is a
change to the command line arguments. The example below will run 1000 simulations
with each simulation taking 5 seconds.
@@ -417,57 +747,156 @@
<profile namespace="globus" key="maxNodes">2</profile>
-----
+////
-Plotting
-~~~~~~~~
-Each part directory contains a file called plot.sh that can be used for plotting.
-It generates two image files: cumulativeplot.png and activeplot.png.
-cumulativeplot.png shows overall progress over time, while activeplot.png shows
-the number of active jobs over time. When the worker agents are trasnferring data
-to and from nodes, this time does not get marked as active.
+Plotting run activity
+~~~~~~~~~~~~~~~~~~~~~
+The tutorial `bin` directory in your `PATH` provides a script
+`plot.sh` to plot the progress of a Swift script. It generates two
+image files: `activeplot.png`, which shows the number of active jobs
+over time, and `cumulativeplot.png`, which shows the total number of
+app calls completed as the Swift script progresses.
+
After each swift run, a log file will be created called
partNN-<YYYYmmdd>-<hhmm>-<random>.log. Once you have identified the
-log file name, run the command `./plot.sh <logfile>` (where logfile is
-the most recent Swift run log) to generate the plots. For example:
+log file name, run the command `./plot.sh` <logfile>` (where logfile
+is the most recent Swift run log) to generate the plots for that
+specific run. For example:
-----
-$ ./plot.sh <logfile>
+$ ls -lt *.log | head
+-rw-r--r-- 1 p01532 61532 2237693 Aug 26 12:45 p4-20130826-1244-kmos0d87.log
+-rw-r--r-- 1 p01532 61532 1008 Aug 26 12:44 swift.log
+-rw-r--r-- 1 p01532 61532 5345345 Aug 26 12:44 p4-20130826-1243-10u2qdbd.log
+-rw-r--r-- 1 p01532 61532 357687 Aug 26 12:00 p4-20130826-1159-j01p4lu0.log
+...
+$ plot.sh p4-20130826-1244-kmos0d87.log
-----
+This yields plots like:
+
image::activeplot.png[width=700,align=center]
image::cumulativeplot.png[width=700,align=center]
+NOTE: Because systems like Raven are often firewalled, you may need to
+use scp to pull these image files back to a system on which you can
+view them with a browser or preview tool.
-You can then scp the two resulting plot images to another host to view them. Eg, from a Mac, do:
------
-$ scp ...
-$ open *.png
-----
-
Part 5: Controlling the compute-node pools where applications run
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-p5.swift introduces a postprocessing step. After all the parallel
-simulations have completed, the files created by simulation.sh will be
-averaged by stats.sh. This is similar to p3, but all app invocations
-are done on remote nodes with Swift managing file transfers.
+In this section we'll use the script `p5.swift`, very similar to
+`p4.swift` of the prior section, to show how we can route apps to
+specific sites, and also how we can make multiple pools of resources
+(on the same or on different computer systems) available to run a
+single Swift script.
+////
image::part05.png[align="center"]
.p5.swift
----
-sys::[cat -n ../part05/p5.swift]
+sys::[cat ../part05/p5.swift]
----
+////
-To run:
+First, lets specify that the analysis app `stats.sh` should be run on
+the local login node instead of on the cluster. This is done simply by
+change the site name field of the analyze app in the apps file:
+
+-----
+$ cat apps
+raven simulate simulate.sh
+localhost stats stats.sh
+-----
+
+Running this with `swift p5.swift` we see:
+-----
+$ grep "on node:" output/*.log
+output/average.log:Running on node: raven
+output/sim_0.log:Running on node: nid00029
+output/sim_1.log:Running on node: nid00029
+output/sim_2.log:Running on node: nid00029
+output/sim_3.log:Running on node: nid00029
+output/sim_4.log:Running on node: nid00029
+output/sim_5.log:Running on node: nid00029
+output/sim_6.log:Running on node: nid00029
+output/sim_7.log:Running on node: nid00029
+output/sim_8.log:Running on node: nid00029
+output/sim_9.log:Running on node: nid00029
+-----
+
+
+Now lets make further use of Swift's ability to route specific apps to
+specific pools of resources. The Cray Raven system has two node types,
+XE6 32-core 2 x IL-16, and XK7 16-core 1 x IL-16 plus one GPU. Each
+"pool" of nodes has different queue characteristics. We can define
+these differences to Swift as two separate pools, and then spread the
+load of executing a large ensemble of simulations across all the
+pools. (And we'll continue to run the analysis script on a third pool,
+comprising the single login host.)
+
+We use the following `apps` file:
+-----
+$ cat multipools
+$ cat multipools
+raven simulate simulate.sh
+ravenGPU simulate simulate.sh
+localhost stats stats.sh
----
-$ swift p5.swift
-----
+and we adjust the sites file to specify 4-node jobs in the Raven pool:
+-----
+ <profile namespace="globus" key="maxNodes">4</profile>
+ <profile namespace="globus" key="nodeGranularity">4</profile>
+-----
+This results in these PBS jobs:
+-----
+p01532 at raven:~> qstat -u $USER
+Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
+--------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
+288687.sdb p01532 small B0827-3406 9919 4 128 -- 00:59 R 00:00
+288688.sdb p01532 gpu_node B0827-3406 9931 6 96 -- 00:59 R 00:00
+-----
+... and achieves the following parallelism (of about 224 concurrent app tasks):
+-----
+$ swift -tc.file multipools p5.swift -nsim=1000 -steps=3
+Swift 0.94.1 RC2 swift-r6895 cog-r3765
+RunID: 20130827-1829-o3h6mht5
+Progress: time: Tue, 27 Aug 2013 18:29:31 -0500
+Progress: time: Tue, 27 Aug 2013 18:29:32 -0500 Initializing:997
+Progress: time: Tue, 27 Aug 2013 18:29:34 -0500 Selecting site:774 Submitting:225 Submitted:1
+Progress: time: Tue, 27 Aug 2013 18:29:35 -0500 Selecting site:774 Stage in:1 Submitted:225
+Progress: time: Tue, 27 Aug 2013 18:29:36 -0500 Selecting site:774 Stage in:1 Submitted:37 Active:188
+Progress: time: Tue, 27 Aug 2013 18:29:39 -0500 Selecting site:774 Submitted:2 Active:223 Stage out:1
+Progress: time: Tue, 27 Aug 2013 18:29:40 -0500 Selecting site:750 Submitted:17 Active:208 Stage out:1 Finished successfully:24
+Progress: time: Tue, 27 Aug 2013 18:29:41 -0500 Selecting site:640 Stage in:1 Submitted:51 Active:174 Finished successfully:134
+Progress: time: Tue, 27 Aug 2013 18:29:42 -0500 Selecting site:551 Submitted:11 Active:214 Stage out:1 Finished successfully:223
+Progress: time: Tue, 27 Aug 2013 18:29:43 -0500 Selecting site:542 Submitted:2 Active:223 Stage out:1 Finished successfully:232
+Progress: time: Tue, 27 Aug 2013 18:29:44 -0500 Selecting site:511 Submitting:1 Submitted:19 Active:206 Finished successfully:263
+Progress: time: Tue, 27 Aug 2013 18:29:45 -0500 Selecting site:463 Stage in:1 Submitted:43 Active:182 Finished successfully:311
+Progress: time: Tue, 27 Aug 2013 18:29:46 -0500 Selecting site:367 Submitting:1 Submitted:38 Active:186 Stage out:1 Finished successfully:407
+Progress: time: Tue, 27 Aug 2013 18:29:47 -0500 Selecting site:309 Submitted:2 Active:223 Stage out:1 Finished successfully:465
+Progress: time: Tue, 27 Aug 2013 18:29:48 -0500 Selecting site:300 Submitted:2 Active:223 Stage out:1 Finished successfully:474
+Progress: time: Tue, 27 Aug 2013 18:29:50 -0500 Selecting site:259 Submitted:11 Active:214 Stage out:1 Finished successfully:515
+Progress: time: Tue, 27 Aug 2013 18:29:51 -0500 Selecting site:201 Stage in:1 Submitted:39 Active:186 Finished successfully:573
+Progress: time: Tue, 27 Aug 2013 18:29:52 -0500 Selecting site:80 Submitted:42 Active:184 Finished successfully:694
+Progress: time: Tue, 27 Aug 2013 18:29:53 -0500 Selecting site:54 Submitted:2 Active:223 Stage out:1 Finished successfully:720
+Progress: time: Tue, 27 Aug 2013 18:29:54 -0500 Selecting site:32 Submitted:4 Active:220 Stage out:1 Finished successfully:743
+Progress: time: Tue, 27 Aug 2013 18:29:55 -0500 Submitted:3 Active:216 Stage out:1 Finished successfully:780
+Progress: time: Tue, 27 Aug 2013 18:29:56 -0500 Stage in:1 Active:143 Finished successfully:856
+Progress: time: Tue, 27 Aug 2013 18:29:57 -0500 Active:38 Stage out:1 Finished successfully:961
+Progress: time: Tue, 27 Aug 2013 18:29:58 -0500 Active:8 Stage out:1 Finished successfully:991
+Progress: time: Tue, 27 Aug 2013 18:29:59 -0500 Stage out:1 Finished successfully:999
+Progress: time: Tue, 27 Aug 2013 18:30:01 -0500 Stage in:1 Finished successfully:1000
+Progress: time: Tue, 27 Aug 2013 18:30:02 -0500 Active:1 Finished successfully:1000
+Progress: time: Tue, 27 Aug 2013 18:30:06 -0500 Stage out:1 Finished successfully:1000
+Final status: Tue, 27 Aug 2013 18:30:07 -0500 Finished successfully:1001
+-----
Part 6: Specifying more complex workflow patterns
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -479,7 +908,7 @@
.p6.swift
----
-sys::[cat -n ../part06/p6.swift]
+sys::[cat ../part06/p6.swift]
----
@@ -495,3 +924,9 @@
+////
+Since the login host (where we run the `swift` command) and the compute nodes (where our swift script runs its apps) share the same filesystems,
+available on both the login node where we are running the `swift`
+command and on the compute nodes where the apps will execute,
+////
+
Modified: SwiftTutorials/swift-cray-tutorial/doc/asciidoc.css
===================================================================
--- SwiftTutorials/swift-cray-tutorial/doc/asciidoc.css 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/doc/asciidoc.css 2013-08-27 23:40:14 UTC (rev 7013)
@@ -3,8 +3,8 @@
.monospaced, code, pre {
font-family: "Courier New", Courier, monospace;
- font-size: small; /* inherit; */
- color: navy;
+ font-size: medium; /* inherit; */
+ color: black;
padding: 0;
margin: 0;
}
Modified: SwiftTutorials/swift-cray-tutorial/doc/push.sh
===================================================================
--- SwiftTutorials/swift-cray-tutorial/doc/push.sh 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/doc/push.sh 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,3 +1,5 @@
#! /bin/sh
-scp swift-cray-tutorial.html *png login.ci.uchicago.edu:/ci/www/projects/swift/tutorials/cray
+# scp -r swift-cray-tutorial.html images *png login.ci.uchicago.edu:/ci/www/projects/swift/tutorials/cray
+
+tar zcf - --exclude-vcs *html *png images | ssh login.ci.uchicago.edu "cd /ci/www/projects/swift/tutorials/cray; tar zxf -"
Modified: SwiftTutorials/swift-cray-tutorial/part01/apps
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part01/apps 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part01/apps 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,2 +1 @@
localhost simulate simulate.sh
-persistent-coasters simulate simulate.sh
Modified: SwiftTutorials/swift-cray-tutorial/part01/p1.swift
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part01/p1.swift 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part01/p1.swift 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,9 +1,9 @@
type file;
-app (file o) mysim ()
+app (file o) simulation ()
{
simulate stdout=@filename(o);
}
file f <"sim.out">;
-f = mysim();
+f = simulation();
Modified: SwiftTutorials/swift-cray-tutorial/part02/p2.swift
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part02/p2.swift 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part02/p2.swift 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,12 +1,11 @@
type file;
-app (file o) mysim ()
+app (file o) simulation ()
{
simulate stdout=@filename(o);
}
foreach i in [0:9] {
file f <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
- f = mysim();
+ f = simulation();
}
-
Modified: SwiftTutorials/swift-cray-tutorial/part03/apps
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part03/apps 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part03/apps 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,4 +1,2 @@
localhost simulate simulate.sh
localhost stats stats.sh
-persistent-coasters simulate simulate.sh
-persistent-coasters stats stats.sh
Modified: SwiftTutorials/swift-cray-tutorial/part03/p3.swift
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part03/p3.swift 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part03/p3.swift 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,9 +1,8 @@
-
type file;
-app (file o) mysim (int sim_steps, int sim_values)
+app (file o) simulation (int sim_steps, int sim_range, int sim_values)
{
- simulate "--timesteps" sim_steps "--nvalues" sim_values stdout=@filename(o);
+ simulate "--timesteps" sim_steps "--range" sim_range "--nvalues" sim_values stdout=@filename(o);
}
app (file o) analyze (file s[])
@@ -13,16 +12,16 @@
int nsim = @toInt(@arg("nsim","10"));
int steps = @toInt(@arg("steps","1"));
+int range = @toInt(@arg("range","100"));
int values = @toInt(@arg("values","5"));
file sims[];
foreach i in [0:nsim-1] {
file simout <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
- simout = mysim(steps,values);
+ simout = simulation(steps,range,values);
sims[i] = simout;
}
file stats<"output/average.out">;
stats = analyze(sims);
-
Modified: SwiftTutorials/swift-cray-tutorial/part04/apps
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part04/apps 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part04/apps 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1 +1,2 @@
-raven simulation /home/users/p01537/swift-cray-tutorial/app/simulate.sh
+raven simulate simulate.sh
+raven stats stats.sh
Modified: SwiftTutorials/swift-cray-tutorial/part04/p4.swift
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part04/p4.swift 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part04/p4.swift 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,27 +1,29 @@
-
type file;
-# "app" function for the simulation application:
+app (file out, file log) simulation (int sim_steps, int sim_range, int sim_values)
+{
+ simulate "--timesteps" sim_steps "--range" sim_range "--nvalues" sim_values stdout=@out stderr=@log;
+}
-app (file out, file log) simulation (int timesteps, int sim_range)
+app (file out, file log) analyze (file s[])
{
- simulation "-t" timesteps "-r" sim_range stdout=@out stderr=@log;
+ stats @filenames(s) stdout=@out stderr=@log;
}
-# Command line parameters to this script:
+int nsim = @toInt(@arg("nsim", "10"));
+int steps = @toInt(@arg("steps", "1"));
+int range = @toInt(@arg("range", "100"));
+int values = @toInt(@arg("values", "5"));
-int nsim = @toInt(@arg("nsim", "10")); # number of simulation programs to run
-int range = @toInt(@arg("range", "100")); # range of the generated random numbers
+file sims[];
-# Main script and data
-
-int steps=3;
-
-tracef("\n*** Script parameters: nsim=%i steps=%i range=%i \n\n", nsim, steps, range);
-
foreach i in [0:nsim-1] {
file simout <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
file simlog <single_file_mapper; file=@strcat("output/sim_",i,".log")>;
- (simout,simlog) = simulation(steps, range);
+ (simout,simlog) = simulation(steps,range,values);
+ sims[i] = simout;
}
+file stats_out<"output/average.out">;
+file stats_log<"output/average.log">;
+(stats_out, stats_log) = analyze(sims);
Copied: SwiftTutorials/swift-cray-tutorial/part05/OLD.p5.swift (from rev 7007, SwiftTutorials/swift-cray-tutorial/part05/p5.swift)
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part05/OLD.p5.swift (rev 0)
+++ SwiftTutorials/swift-cray-tutorial/part05/OLD.p5.swift 2013-08-27 23:40:14 UTC (rev 7013)
@@ -0,0 +1,37 @@
+type file;
+
+app (file out, file log) simulation (int timesteps, int sim_range, file bias_file, int scale, int sim_count)
+{
+ simulate "-t" timesteps "-r" sim_range "-B" @bias_file "-x" scale "-n" sim_count stdout=@out stderr=@log;
+}
+
+app (file out) analyze (file s[])
+{
+ analyze @filenames(s) stdout=@filename(out);
+}
+
+# Command line params to this script
+
+int nsim = @toInt(@arg("nsim", "10")); # number of simulation programs to run
+int steps = @toInt(@arg("steps", "1")); # number of "steps" each simulation (==seconds of runtime)
+int range = @toInt(@arg("range", "100")); # range of the generated random numbers
+int count = @toInt(@arg("count", "10")); # number of random numbers generated per simulation
+
+# Perform nsim "simulations"
+
+tracef("\n*** Script parameters: nsim=%i steps=%i range=%i count=%i\n\n", nsim, steps, range, count);
+
+file sims[]; # Array of files to hold each simulation output
+file bias<"bias.dat">; # Input data file to "bias" the numbers:
+ # 1 line: scale offset ( N = n*scale + offset)
+foreach i in [0:nsim-1] {
+ file simout <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
+ file simlog <single_file_mapper; file=@strcat("output/sim_",i,".log")>;
+ (simout, simlog) = simulation(steps, range, bias, 100000, count);
+ sims[i] = simout;
+}
+
+# Generate "analysis" file containing average of all "simulations"
+
+file stats<"output/stats.out">;
+stats = analyze(sims);
Copied: SwiftTutorials/swift-cray-tutorial/part05/OLDbias.dat (from rev 7007, SwiftTutorials/swift-cray-tutorial/part05/bias.dat)
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part05/OLDbias.dat (rev 0)
+++ SwiftTutorials/swift-cray-tutorial/part05/OLDbias.dat 2013-08-27 23:40:14 UTC (rev 7013)
@@ -0,0 +1,1000 @@
+ 301
+ 439
+ 591
+ 484
+ 300
+ 459
+ 395
+ 351
+ 375
+ 131
+ 144
+ 12
+ 465
+ 682
+ 245
+ 140
+ 862
+ 772
+ 52
+ 728
+ 696
+ 801
+ 487
+ 501
+ 273
+ 869
+ 976
+ 731
+ 634
+ 594
+ 944
+ 973
+ 521
+ 260
+ 997
+ 165
+ 240
+ 844
+ 30
+ 145
+ 29
+ 185
+ 19
+ 332
+ 932
+ 740
+ 750
+ 774
+ 882
+ 258
+ 64
+ 988
+ 980
+ 627
+ 114
+ 722
+ 522
+ 333
+ 201
+ 737
+ 773
+ 226
+ 99
+ 705
+ 435
+ 560
+ 311
+ 96
+ 538
+ 176
+ 831
+ 993
+ 761
+ 917
+ 311
+ 983
+ 259
+ 494
+ 713
+ 503
+ 580
+ 23
+ 687
+ 394
+ 935
+ 386
+ 420
+ 598
+ 424
+ 954
+ 92
+ 528
+ 43
+ 384
+ 531
+ 494
+ 335
+ 913
+ 140
+ 271
+ 817
+ 101
+ 996
+ 742
+ 36
+ 592
+ 321
+ 499
+ 579
+ 454
+ 333
+ 276
+ 360
+ 159
+ 909
+ 791
+ 913
+ 665
+ 862
+ 663
+ 455
+ 39
+ 415
+ 798
+ 222
+ 758
+ 178
+ 656
+ 144
+ 450
+ 182
+ 507
+ 269
+ 839
+ 911
+ 400
+ 506
+ 602
+ 69
+ 729
+ 697
+ 242
+ 39
+ 557
+ 756
+ 677
+ 403
+ 930
+ 660
+ 730
+ 974
+ 23
+ 227
+ 67
+ 465
+ 950
+ 819
+ 100
+ 858
+ 123
+ 979
+ 610
+ 428
+ 623
+ 12
+ 369
+ 96
+ 86
+ 531
+ 777
+ 809
+ 9
+ 589
+ 267
+ 957
+ 722
+ 352
+ 655
+ 145
+ 848
+ 232
+ 752
+ 958
+ 902
+ 206
+ 165
+ 52
+ 452
+ 185
+ 109
+ 676
+ 736
+ 803
+ 893
+ 163
+ 403
+ 573
+ 121
+ 992
+ 818
+ 571
+ 304
+ 822
+ 390
+ 192
+ 174
+ 571
+ 648
+ 924
+ 817
+ 334
+ 791
+ 561
+ 679
+ 457
+ 549
+ 339
+ 264
+ 316
+ 338
+ 726
+ 471
+ 144
+ 748
+ 778
+ 851
+ 432
+ 260
+ 731
+ 436
+ 376
+ 631
+ 373
+ 680
+ 223
+ 169
+ 916
+ 252
+ 236
+ 2
+ 565
+ 749
+ 206
+ 788
+ 801
+ 460
+ 730
+ 795
+ 17
+ 992
+ 847
+ 229
+ 187
+ 690
+ 111
+ 731
+ 754
+ 677
+ 845
+ 707
+ 815
+ 258
+ 526
+ 475
+ 559
+ 970
+ 440
+ 682
+ 471
+ 378
+ 560
+ 821
+ 851
+ 819
+ 106
+ 624
+ 766
+ 628
+ 362
+ 149
+ 523
+ 215
+ 207
+ 446
+ 783
+ 845
+ 556
+ 273
+ 921
+ 519
+ 81
+ 598
+ 440
+ 899
+ 218
+ 623
+ 861
+ 475
+ 789
+ 643
+ 165
+ 526
+ 705
+ 803
+ 907
+ 725
+ 814
+ 306
+ 39
+ 795
+ 144
+ 986
+ 79
+ 726
+ 664
+ 99
+ 71
+ 516
+ 948
+ 601
+ 67
+ 111
+ 856
+ 915
+ 747
+ 621
+ 360
+ 93
+ 787
+ 880
+ 694
+ 5
+ 982
+ 336
+ 426
+ 346
+ 639
+ 10
+ 427
+ 757
+ 981
+ 102
+ 325
+ 305
+ 691
+ 619
+ 398
+ 686
+ 922
+ 674
+ 349
+ 580
+ 165
+ 321
+ 492
+ 432
+ 329
+ 477
+ 894
+ 444
+ 57
+ 577
+ 323
+ 278
+ 179
+ 421
+ 614
+ 616
+ 436
+ 892
+ 432
+ 558
+ 140
+ 517
+ 975
+ 198
+ 805
+ 422
+ 745
+ 250
+ 482
+ 958
+ 538
+ 177
+ 791
+ 489
+ 932
+ 620
+ 138
+ 978
+ 689
+ 192
+ 742
+ 626
+ 946
+ 106
+ 31
+ 178
+ 205
+ 285
+ 576
+ 105
+ 179
+ 307
+ 830
+ 576
+ 679
+ 231
+ 454
+ 833
+ 116
+ 160
+ 616
+ 250
+ 760
+ 205
+ 74
+ 989
+ 576
+ 843
+ 931
+ 267
+ 849
+ 108
+ 850
+ 937
+ 746
+ 607
+ 35
+ 539
+ 550
+ 553
+ 886
+ 459
+ 786
+ 720
+ 650
+ 735
+ 126
+ 100
+ 297
+ 303
+ 921
+ 105
+ 46
+ 954
+ 663
+ 568
+ 478
+ 886
+ 817
+ 690
+ 261
+ 642
+ 511
+ 967
+ 727
+ 492
+ 4
+ 810
+ 320
+ 338
+ 498
+ 842
+ 454
+ 366
+ 316
+ 802
+ 641
+ 293
+ 544
+ 957
+ 768
+ 553
+ 692
+ 749
+ 944
+ 19
+ 762
+ 991
+ 234
+ 247
+ 556
+ 625
+ 189
+ 725
+ 818
+ 764
+ 636
+ 370
+ 138
+ 161
+ 853
+ 309
+ 613
+ 250
+ 75
+ 728
+ 230
+ 248
+ 562
+ 110
+ 879
+ 993
+ 626
+ 308
+ 124
+ 593
+ 169
+ 125
+ 304
+ 143
+ 312
+ 525
+ 639
+ 492
+ 377
+ 153
+ 97
+ 90
+ 1
+ 483
+ 139
+ 277
+ 7
+ 707
+ 977
+ 513
+ 458
+ 95
+ 208
+ 149
+ 293
+ 690
+ 456
+ 558
+ 245
+ 327
+ 500
+ 222
+ 776
+ 863
+ 322
+ 363
+ 892
+ 414
+ 442
+ 283
+ 627
+ 656
+ 785
+ 468
+ 371
+ 657
+ 840
+ 449
+ 500
+ 522
+ 337
+ 784
+ 475
+ 667
+ 345
+ 151
+ 541
+ 843
+ 285
+ 270
+ 684
+ 381
+ 608
+ 731
+ 492
+ 233
+ 540
+ 185
+ 975
+ 422
+ 199
+ 595
+ 242
+ 995
+ 892
+ 509
+ 916
+ 493
+ 549
+ 588
+ 203
+ 653
+ 276
+ 808
+ 245
+ 277
+ 827
+ 724
+ 118
+ 566
+ 406
+ 927
+ 498
+ 727
+ 309
+ 165
+ 412
+ 28
+ 937
+ 797
+ 556
+ 322
+ 59
+ 741
+ 675
+ 89
+ 877
+ 283
+ 518
+ 509
+ 6
+ 486
+ 421
+ 644
+ 258
+ 920
+ 623
+ 725
+ 859
+ 88
+ 773
+ 499
+ 973
+ 661
+ 947
+ 347
+ 679
+ 935
+ 717
+ 65
+ 265
+ 388
+ 740
+ 252
+ 249
+ 969
+ 26
+ 149
+ 338
+ 913
+ 605
+ 639
+ 649
+ 914
+ 890
+ 522
+ 621
+ 779
+ 470
+ 321
+ 171
+ 304
+ 425
+ 626
+ 378
+ 491
+ 925
+ 900
+ 705
+ 830
+ 631
+ 844
+ 274
+ 539
+ 511
+ 664
+ 777
+ 251
+ 190
+ 967
+ 569
+ 140
+ 327
+ 863
+ 525
+ 368
+ 540
+ 475
+ 716
+ 189
+ 442
+ 729
+ 112
+ 726
+ 505
+ 678
+ 323
+ 128
+ 983
+ 660
+ 853
+ 326
+ 312
+ 378
+ 544
+ 278
+ 127
+ 322
+ 360
+ 645
+ 233
+ 665
+ 3
+ 788
+ 698
+ 342
+ 93
+ 214
+ 976
+ 756
+ 677
+ 375
+ 947
+ 363
+ 999
+ 499
+ 139
+ 245
+ 745
+ 830
+ 798
+ 135
+ 83
+ 902
+ 779
+ 617
+ 636
+ 753
+ 867
+ 724
+ 743
+ 924
+ 480
+ 836
+ 409
+ 100
+ 73
+ 298
+ 711
+ 35
+ 289
+ 381
+ 361
+ 901
+ 640
+ 919
+ 708
+ 562
+ 592
+ 26
+ 471
+ 395
+ 728
+ 433
+ 69
+ 856
+ 550
+ 225
+ 714
+ 999
+ 936
+ 266
+ 81
+ 343
+ 123
+ 37
+ 740
+ 593
+ 905
+ 715
+ 481
+ 265
+ 357
+ 338
+ 150
+ 875
+ 938
+ 603
+ 98
+ 245
+ 721
+ 373
+ 901
+ 795
+ 325
+ 955
+ 448
+ 816
+ 753
+ 850
+ 462
+ 628
+ 220
+ 694
+ 423
+ 241
+ 718
+ 730
+ 682
+ 944
+ 408
+ 151
+ 10
+ 826
+ 26
+ 269
+ 439
+ 973
+ 842
+ 884
+ 103
+ 792
+ 435
+ 675
+ 442
+ 955
+ 992
+ 256
+ 206
+ 666
+ 637
+ 725
+ 52
+ 21
+ 108
+ 622
+ 943
+ 583
+ 919
+ 796
+ 80
+ 52
+ 678
+ 845
+ 155
+ 331
+ 909
+ 888
+ 945
+ 979
+ 967
+ 104
+ 446
+ 807
+ 572
+ 354
+ 639
+ 68
+ 813
+ 11
+ 129
+ 647
+ 743
+ 897
+ 600
+ 992
+ 20
+ 896
+ 398
+ 16
+ 434
+ 662
+ 86
+ 200
+ 211
+ 287
+ 485
+ 670
+ 446
+ 495
+ 407
+ 916
+ 395
+ 920
+ 959
+ 817
+ 524
+ 146
+ 378
+ 849
+ 457
+ 536
+ 582
+ 608
+ 686
+ 127
+ 346
+ 934
+ 950
+ 420
+ 294
+ 439
+ 688
+ 953
+ 66
+ 603
+ 875
+ 372
+ 355
+ 105
+ 405
+ 371
+ 246
+ 829
+ 146
+ 403
+ 811
+ 484
+ 266
+ 181
+ 496
+ 25
+ 189
+ 806
+ 1
+ 171
+ 737
+ 381
+ 720
+ 321
+ 862
+ 7
+ 177
+ 838
+ 502
+ 708
+ 819
+ 205
+ 256
+ 166
+ 712
+ 492
+ 716
+ 130
+ 562
+ 308
+ 365
+ 694
+ 372
+ 917
+ 165
+ 366
+ 412
+ 480
+ 681
+ 824
+ 306
+ 970
+ 160
+ 555
+ 336
+ 61
+ 816
+ 611
+ 432
+ 536
+ 830
+ 171
+ 793
+ 33
+ 364
+ 998
+ 814
+ 952
+ 89
+ 24
+ 653
+ 651
+ 8
+ 789
+ 807
+ 869
+ 332
+ 897
+ 499
+ 701
+ 354
+ 688
+ 339
+ 93
+ 822
+ 801
+ 758
+ 915
+ 546
+ 520
+ 469
+ 512
+ 911
+ 482
+ 387
+ 187
Modified: SwiftTutorials/swift-cray-tutorial/part05/apps
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part05/apps 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part05/apps 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,2 +1,2 @@
-raven simulation /home/users/p01537/swift-cray-tutorial/app/simulate.sh
-raven analyze /home/users/p01537/swift-cray-tutorial/app/stats.sh
+raven simulate simulate.sh
+raven stats stats.sh
Deleted: SwiftTutorials/swift-cray-tutorial/part05/bias.dat
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part05/bias.dat 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part05/bias.dat 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,1000 +0,0 @@
- 301
- 439
- 591
- 484
- 300
- 459
- 395
- 351
- 375
- 131
- 144
- 12
- 465
- 682
- 245
- 140
- 862
- 772
- 52
- 728
- 696
- 801
- 487
- 501
- 273
- 869
- 976
- 731
- 634
- 594
- 944
- 973
- 521
- 260
- 997
- 165
- 240
- 844
- 30
- 145
- 29
- 185
- 19
- 332
- 932
- 740
- 750
- 774
- 882
- 258
- 64
- 988
- 980
- 627
- 114
- 722
- 522
- 333
- 201
- 737
- 773
- 226
- 99
- 705
- 435
- 560
- 311
- 96
- 538
- 176
- 831
- 993
- 761
- 917
- 311
- 983
- 259
- 494
- 713
- 503
- 580
- 23
- 687
- 394
- 935
- 386
- 420
- 598
- 424
- 954
- 92
- 528
- 43
- 384
- 531
- 494
- 335
- 913
- 140
- 271
- 817
- 101
- 996
- 742
- 36
- 592
- 321
- 499
- 579
- 454
- 333
- 276
- 360
- 159
- 909
- 791
- 913
- 665
- 862
- 663
- 455
- 39
- 415
- 798
- 222
- 758
- 178
- 656
- 144
- 450
- 182
- 507
- 269
- 839
- 911
- 400
- 506
- 602
- 69
- 729
- 697
- 242
- 39
- 557
- 756
- 677
- 403
- 930
- 660
- 730
- 974
- 23
- 227
- 67
- 465
- 950
- 819
- 100
- 858
- 123
- 979
- 610
- 428
- 623
- 12
- 369
- 96
- 86
- 531
- 777
- 809
- 9
- 589
- 267
- 957
- 722
- 352
- 655
- 145
- 848
- 232
- 752
- 958
- 902
- 206
- 165
- 52
- 452
- 185
- 109
- 676
- 736
- 803
- 893
- 163
- 403
- 573
- 121
- 992
- 818
- 571
- 304
- 822
- 390
- 192
- 174
- 571
- 648
- 924
- 817
- 334
- 791
- 561
- 679
- 457
- 549
- 339
- 264
- 316
- 338
- 726
- 471
- 144
- 748
- 778
- 851
- 432
- 260
- 731
- 436
- 376
- 631
- 373
- 680
- 223
- 169
- 916
- 252
- 236
- 2
- 565
- 749
- 206
- 788
- 801
- 460
- 730
- 795
- 17
- 992
- 847
- 229
- 187
- 690
- 111
- 731
- 754
- 677
- 845
- 707
- 815
- 258
- 526
- 475
- 559
- 970
- 440
- 682
- 471
- 378
- 560
- 821
- 851
- 819
- 106
- 624
- 766
- 628
- 362
- 149
- 523
- 215
- 207
- 446
- 783
- 845
- 556
- 273
- 921
- 519
- 81
- 598
- 440
- 899
- 218
- 623
- 861
- 475
- 789
- 643
- 165
- 526
- 705
- 803
- 907
- 725
- 814
- 306
- 39
- 795
- 144
- 986
- 79
- 726
- 664
- 99
- 71
- 516
- 948
- 601
- 67
- 111
- 856
- 915
- 747
- 621
- 360
- 93
- 787
- 880
- 694
- 5
- 982
- 336
- 426
- 346
- 639
- 10
- 427
- 757
- 981
- 102
- 325
- 305
- 691
- 619
- 398
- 686
- 922
- 674
- 349
- 580
- 165
- 321
- 492
- 432
- 329
- 477
- 894
- 444
- 57
- 577
- 323
- 278
- 179
- 421
- 614
- 616
- 436
- 892
- 432
- 558
- 140
- 517
- 975
- 198
- 805
- 422
- 745
- 250
- 482
- 958
- 538
- 177
- 791
- 489
- 932
- 620
- 138
- 978
- 689
- 192
- 742
- 626
- 946
- 106
- 31
- 178
- 205
- 285
- 576
- 105
- 179
- 307
- 830
- 576
- 679
- 231
- 454
- 833
- 116
- 160
- 616
- 250
- 760
- 205
- 74
- 989
- 576
- 843
- 931
- 267
- 849
- 108
- 850
- 937
- 746
- 607
- 35
- 539
- 550
- 553
- 886
- 459
- 786
- 720
- 650
- 735
- 126
- 100
- 297
- 303
- 921
- 105
- 46
- 954
- 663
- 568
- 478
- 886
- 817
- 690
- 261
- 642
- 511
- 967
- 727
- 492
- 4
- 810
- 320
- 338
- 498
- 842
- 454
- 366
- 316
- 802
- 641
- 293
- 544
- 957
- 768
- 553
- 692
- 749
- 944
- 19
- 762
- 991
- 234
- 247
- 556
- 625
- 189
- 725
- 818
- 764
- 636
- 370
- 138
- 161
- 853
- 309
- 613
- 250
- 75
- 728
- 230
- 248
- 562
- 110
- 879
- 993
- 626
- 308
- 124
- 593
- 169
- 125
- 304
- 143
- 312
- 525
- 639
- 492
- 377
- 153
- 97
- 90
- 1
- 483
- 139
- 277
- 7
- 707
- 977
- 513
- 458
- 95
- 208
- 149
- 293
- 690
- 456
- 558
- 245
- 327
- 500
- 222
- 776
- 863
- 322
- 363
- 892
- 414
- 442
- 283
- 627
- 656
- 785
- 468
- 371
- 657
- 840
- 449
- 500
- 522
- 337
- 784
- 475
- 667
- 345
- 151
- 541
- 843
- 285
- 270
- 684
- 381
- 608
- 731
- 492
- 233
- 540
- 185
- 975
- 422
- 199
- 595
- 242
- 995
- 892
- 509
- 916
- 493
- 549
- 588
- 203
- 653
- 276
- 808
- 245
- 277
- 827
- 724
- 118
- 566
- 406
- 927
- 498
- 727
- 309
- 165
- 412
- 28
- 937
- 797
- 556
- 322
- 59
- 741
- 675
- 89
- 877
- 283
- 518
- 509
- 6
- 486
- 421
- 644
- 258
- 920
- 623
- 725
- 859
- 88
- 773
- 499
- 973
- 661
- 947
- 347
- 679
- 935
- 717
- 65
- 265
- 388
- 740
- 252
- 249
- 969
- 26
- 149
- 338
- 913
- 605
- 639
- 649
- 914
- 890
- 522
- 621
- 779
- 470
- 321
- 171
- 304
- 425
- 626
- 378
- 491
- 925
- 900
- 705
- 830
- 631
- 844
- 274
- 539
- 511
- 664
- 777
- 251
- 190
- 967
- 569
- 140
- 327
- 863
- 525
- 368
- 540
- 475
- 716
- 189
- 442
- 729
- 112
- 726
- 505
- 678
- 323
- 128
- 983
- 660
- 853
- 326
- 312
- 378
- 544
- 278
- 127
- 322
- 360
- 645
- 233
- 665
- 3
- 788
- 698
- 342
- 93
- 214
- 976
- 756
- 677
- 375
- 947
- 363
- 999
- 499
- 139
- 245
- 745
- 830
- 798
- 135
- 83
- 902
- 779
- 617
- 636
- 753
- 867
- 724
- 743
- 924
- 480
- 836
- 409
- 100
- 73
- 298
- 711
- 35
- 289
- 381
- 361
- 901
- 640
- 919
- 708
- 562
- 592
- 26
- 471
- 395
- 728
- 433
- 69
- 856
- 550
- 225
- 714
- 999
- 936
- 266
- 81
- 343
- 123
- 37
- 740
- 593
- 905
- 715
- 481
- 265
- 357
- 338
- 150
- 875
- 938
- 603
- 98
- 245
- 721
- 373
- 901
- 795
- 325
- 955
- 448
- 816
- 753
- 850
- 462
- 628
- 220
- 694
- 423
- 241
- 718
- 730
- 682
- 944
- 408
- 151
- 10
- 826
- 26
- 269
- 439
- 973
- 842
- 884
- 103
- 792
- 435
- 675
- 442
- 955
- 992
- 256
- 206
- 666
- 637
- 725
- 52
- 21
- 108
- 622
- 943
- 583
- 919
- 796
- 80
- 52
- 678
- 845
- 155
- 331
- 909
- 888
- 945
- 979
- 967
- 104
- 446
- 807
- 572
- 354
- 639
- 68
- 813
- 11
- 129
- 647
- 743
- 897
- 600
- 992
- 20
- 896
- 398
- 16
- 434
- 662
- 86
- 200
- 211
- 287
- 485
- 670
- 446
- 495
- 407
- 916
- 395
- 920
- 959
- 817
- 524
- 146
- 378
- 849
- 457
- 536
- 582
- 608
- 686
- 127
- 346
- 934
- 950
- 420
- 294
- 439
- 688
- 953
- 66
- 603
- 875
- 372
- 355
- 105
- 405
- 371
- 246
- 829
- 146
- 403
- 811
- 484
- 266
- 181
- 496
- 25
- 189
- 806
- 1
- 171
- 737
- 381
- 720
- 321
- 862
- 7
- 177
- 838
- 502
- 708
- 819
- 205
- 256
- 166
- 712
- 492
- 716
- 130
- 562
- 308
- 365
- 694
- 372
- 917
- 165
- 366
- 412
- 480
- 681
- 824
- 306
- 970
- 160
- 555
- 336
- 61
- 816
- 611
- 432
- 536
- 830
- 171
- 793
- 33
- 364
- 998
- 814
- 952
- 89
- 24
- 653
- 651
- 8
- 789
- 807
- 869
- 332
- 897
- 499
- 701
- 354
- 688
- 339
- 93
- 822
- 801
- 758
- 915
- 546
- 520
- 469
- 512
- 911
- 482
- 387
- 187
Added: SwiftTutorials/swift-cray-tutorial/part05/multipools
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part05/multipools (rev 0)
+++ SwiftTutorials/swift-cray-tutorial/part05/multipools 2013-08-27 23:40:14 UTC (rev 7013)
@@ -0,0 +1,3 @@
+raven simulate simulate.sh
+ravenGPU simulate simulate.sh
+localhost stats stats.sh
Deleted: SwiftTutorials/swift-cray-tutorial/part05/p5.swift
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part05/p5.swift 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part05/p5.swift 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,37 +0,0 @@
-type file;
-
-app (file out, file log) simulation (int timesteps, int sim_range, file bias_file, int scale, int sim_count)
-{
- simulation "-t" timesteps "-r" sim_range "-B" @bias_file "-x" scale "-n" sim_count stdout=@out stderr=@log;
-}
-
-app (file out) analyze (file s[])
-{
- analyze @filenames(s) stdout=@filename(out);
-}
-
-# Command line params to this script
-
-int nsim = @toInt(@arg("nsim", "10")); # number of simulation programs to run
-int steps = @toInt(@arg("steps", "1")); # number of "steps" each simulation (==seconds of runtime)
-int range = @toInt(@arg("range", "100")); # range of the generated random numbers
-int count = @toInt(@arg("count", "10")); # number of random numbers generated per simulation
-
-# Perform nsim "simulations"
-
-tracef("\n*** Script parameters: nsim=%i steps=%i range=%i count=%i\n\n", nsim, steps, range, count);
-
-file sims[]; # Array of files to hold each simulation output
-file bias<"bias.dat">; # Input data file to "bias" the numbers:
- # 1 line: scale offset ( N = n*scale + offset)
-foreach i in [0:nsim-1] {
- file simout <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
- file simlog <single_file_mapper; file=@strcat("output/sim_",i,".log")>;
- (simout, simlog) = simulation(steps, range, bias, 100000, count);
- sims[i] = simout;
-}
-
-# Generate "analysis" file containing average of all "simulations"
-
-file stats<"output/stats.out">;
-stats = analyze(sims);
Copied: SwiftTutorials/swift-cray-tutorial/part05/p5.swift (from rev 7007, SwiftTutorials/swift-cray-tutorial/part04/p4.swift)
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part05/p5.swift (rev 0)
+++ SwiftTutorials/swift-cray-tutorial/part05/p5.swift 2013-08-27 23:40:14 UTC (rev 7013)
@@ -0,0 +1,29 @@
+type file;
+
+app (file out, file log) simulation (int sim_steps, int sim_range, int sim_values)
+{
+ simulate "--timesteps" sim_steps "--range" sim_range "--nvalues" sim_values stdout=@out stderr=@log;
+}
+
+app (file out, file log) analyze (file s[])
+{
+ stats @filenames(s) stdout=@out stderr=@log;
+}
+
+int nsim = @toInt(@arg("nsim", "10"));
+int steps = @toInt(@arg("steps", "1"));
+int range = @toInt(@arg("range", "100"));
+int values = @toInt(@arg("values", "5"));
+
+file sims[];
+
+foreach i in [0:nsim-1] {
+ file simout <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
+ file simlog <single_file_mapper; file=@strcat("output/sim_",i,".log")>;
+ (simout,simlog) = simulation(steps,range,values);
+ sims[i] = simout;
+}
+
+file stats_out<"output/average.out">;
+file stats_log<"output/average.log">;
+(stats_out, stats_log) = analyze(sims);
Modified: SwiftTutorials/swift-cray-tutorial/part06/apps
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part06/apps 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part06/apps 2013-08-27 23:40:14 UTC (rev 7013)
@@ -1,4 +1,4 @@
-raven simulation /home/users/p01537/swift-cray-tutorial/app/simulate.sh
-raven analyze /home/users/p01537/swift-cray-tutorial/app/stats.sh
-raven genseed /home/users/p01537/swift-cray-tutorial/app/simulate.sh
-raven genbias /home/users/p01537/swift-cray-tutorial/app/simulate.sh
+raven simulate /home/users/p01537/swift-cray-tutorial/app/simulate.sh
+raven analyze /home/users/p01537/swift-cray-tutorial/app/stats.sh
+raven genseed /home/users/p01537/swift-cray-tutorial/app/simulate.sh
+raven genbias /home/users/p01537/swift-cray-tutorial/app/simulate.sh
Modified: SwiftTutorials/swift-cray-tutorial/part06/p6.swift
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part06/p6.swift 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/part06/p6.swift 2013-08-27 23:40:14 UTC (rev 7013)
@@ -15,7 +15,7 @@
app (file out, file log) simulation (int timesteps, int sim_range,
file bias_file, int scale, int sim_count)
{
- simulation "-t" timesteps "-r" sim_range "-B" @bias_file "-x" scale
+ simulate "-t" timesteps "-r" sim_range "-B" @bias_file "-x" scale
"-n" sim_count stdout=@out stderr=@log;
}
Modified: SwiftTutorials/swift-cray-tutorial/setup.sh
===================================================================
--- SwiftTutorials/swift-cray-tutorial/setup.sh 2013-08-27 23:39:24 UTC (rev 7012)
+++ SwiftTutorials/swift-cray-tutorial/setup.sh 2013-08-27 23:40:14 UTC (rev 7013)
@@ -71,7 +71,7 @@
END
-cat >sites.raven<<END
+cat >gen.sites <<END
<?xml version="1.0" encoding="UTF-8"?>
<config xmlns="http://swift-lang.org/sites">
@@ -85,12 +85,17 @@
<pool handle="raven">
<execution provider="coaster" jobmanager="local:pbs" URL="local:01"/>
+ <profile namespace="env" key="PATHPREFIX">{env.PWD}/../app</profile>
<profile namespace="globus" key="jobsPerNode">32</profile>
<profile namespace="globus" key="queue">small</profile>
<profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=32</profile>
<profile namespace="globus" key="maxWallTime">00:01:00</profile>
+ <profile namespace="globus" key="maxTime">3600</profile>
+ <profile namespace="globus" key="lowOverAllocation">10000</profile>
+ <profile namespace="globus" key="highOverAllocation">10000</profile>
<profile namespace="globus" key="slots">2</profile>
<profile namespace="globus" key="maxNodes">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
<profile namespace="karajan" key="jobThrottle">3.20</profile>
<profile namespace="karajan" key="initialScore">10000</profile>
<workdirectory>/lus/scratch/{env.USER}/swiftwork</workdirectory>
@@ -99,10 +104,14 @@
<pool handle="ravenMED">
<execution provider="coaster" jobmanager="local:pbs" URL="local:02"/>
+ <profile namespace="env" key="PATHPREFIX">{env.PWD}/../app</profile>
<profile namespace="globus" key="queue">medium</profile>
<profile namespace="globus" key="jobsPerNode">32</profile>
<profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=32</profile>
<profile namespace="globus" key="maxWallTime">00:01:00</profile>
+ <profile namespace="globus" key="maxTime">3600</profile>
+ <profile namespace="globus" key="lowOverAllocation">10000</profile>
+ <profile namespace="globus" key="highOverAllocation">10000</profile>
<profile namespace="globus" key="slots">1</profile>
<profile namespace="globus" key="maxNodes">8</profile>
<profile namespace="globus" key="nodeGranularity">8</profile>
@@ -114,10 +123,14 @@
<pool handle="ravenGPU">
<execution provider="coaster" jobmanager="local:pbs" URL="local:03"/>
+ <profile namespace="env" key="PATHPREFIX">{env.PWD}/../app</profile>
<profile namespace="globus" key="queue">gpu_nodes</profile>
<profile namespace="globus" key="jobsPerNode">16</profile>
<profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=16</profile>
<profile namespace="globus" key="maxWallTime">00:01:00</profile>
+ <profile namespace="globus" key="maxTime">3600</profile>
+ <profile namespace="globus" key="lowOverAllocation">10000</profile>
+ <profile namespace="globus" key="highOverAllocation">10000</profile>
<profile namespace="globus" key="slots">1</profile>
<profile namespace="globus" key="maxNodes">6</profile>
<profile namespace="globus" key="nodeGranularity">6</profile>
@@ -131,10 +144,10 @@
END
for p in 04 05 06; do
- cp sites.raven part${p}/sites.xml
+ cp gen.sites part${p}/sites.xml
done
-rm sites.raven
+rm gen.sites
return
More information about the Swift-commit
mailing list