[Swift-commit] r7014 - in SwiftTutorials/swift-cray-tutorial: . doc part06
wilde at ci.uchicago.edu
wilde at ci.uchicago.edu
Tue Aug 27 19:39:20 CDT 2013
Author: wilde
Date: 2013-08-27 19:39:20 -0500 (Tue, 27 Aug 2013)
New Revision: 7014
Modified:
SwiftTutorials/swift-cray-tutorial/doc/README
SwiftTutorials/swift-cray-tutorial/part06/apps
SwiftTutorials/swift-cray-tutorial/part06/p6.swift
SwiftTutorials/swift-cray-tutorial/setup.sh
Log:
Updates for part06.
Modified: SwiftTutorials/swift-cray-tutorial/doc/README
===================================================================
--- SwiftTutorials/swift-cray-tutorial/doc/README 2013-08-27 23:40:14 UTC (rev 7013)
+++ SwiftTutorials/swift-cray-tutorial/doc/README 2013-08-28 00:39:20 UTC (rev 7014)
@@ -901,8 +901,11 @@
Part 6: Specifying more complex workflow patterns
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-p6.swift build on p5.swift, but adds new apps for generating a random
-seed and a random bias value.
+p6.swift expands the workflow pattern of p4.swift to add additional
+stages to the workflow. Here, we generate a dynamic seed value that
+will be used by all of the simulations, and for each simulation, we
+run an pre-processing application to generate a unique "bias
+file". This pattern is shown below, followed by the Swift script.
image::part06.png[align="center"]
@@ -911,22 +914,121 @@
sys::[cat ../part06/p6.swift]
----
+Note that the workflow is based on data flow dependencies: each simulation depends on the seed value, calculated in these two dependent statements:
+-----
+seedfile = genseed(1);
+int seedval = readData(seedfile);
+-----
+and on the bias file, computed and then consumed in these two dependent statements:
+-----
+ biasfile = genbias(1000, 20);
+ (simout,simlog) = simulation(steps, range, biasfile, 1000000, values);
+-----
-Use the command below to specify the time for each simulation.
+To run:
----
$ cd ../part06
-$ swift p6.swift -steps=3 # each simulation takes 3 seconds
+$ swift p6.swift
----
+The default parameters result in the following execution log:
+-----
+$ swift p6.swift
+Swift 0.94.1 RC2 swift-r6895 cog-r3765
+RunID: 20130827-1917-jvs4gqm5
+Progress: time: Tue, 27 Aug 2013 19:17:56 -0500
+*** Script parameters: nsim=10 range=100 num values=10
+Progress: time: Tue, 27 Aug 2013 19:17:57 -0500 Stage in:1 Submitted:10
+Generated seed=382537
+Progress: time: Tue, 27 Aug 2013 19:17:59 -0500 Active:9 Stage out:1 Finished successfully:11
+Final status: Tue, 27 Aug 2013 19:18:00 -0500 Finished successfully:22
+-----
+which produces the following output:
+-----
+$ ls -lrt output
+total 264
+-rw-r--r-- 1 p01532 61532 9 Aug 27 19:17 seed.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_9.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_8.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_7.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_6.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_5.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_4.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_3.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_2.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_1.dat
+-rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_0.dat
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_9.out
+-rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_9.log
+-rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_8.log
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_7.out
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_6.out
+-rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_6.log
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_5.out
+-rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_5.log
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_4.out
+-rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_4.log
+-rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_1.log
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_8.out
+-rw-r--r-- 1 p01532 61532 14897 Aug 27 19:18 sim_7.log
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_3.out
+-rw-r--r-- 1 p01532 61532 14897 Aug 27 19:18 sim_3.log
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_2.out
+-rw-r--r-- 1 p01532 61532 14898 Aug 27 19:18 sim_2.log
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_1.out
+-rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_0.out
+-rw-r--r-- 1 p01532 61532 14897 Aug 27 19:18 sim_0.log
+-rw-r--r-- 1 p01532 61532 9 Aug 27 19:18 average.out
+-rw-r--r-- 1 p01532 61532 14675 Aug 27 19:18 average.log
+-----
+Each sim_N.out file is the sum of its bias file plus newly "simulated" random output scaled by 1,000,000:
-////
-Since the login host (where we run the `swift` command) and the compute nodes (where our swift script runs its apps) share the same filesystems,
-available on both the login node where we are running the `swift`
-command and on the compute nodes where the apps will execute,
-////
+-----
+$ cat output/bias_0.dat
+ 302
+ 489
+ 81
+ 582
+ 664
+ 290
+ 839
+ 258
+ 506
+ 310
+ 293
+ 508
+ 88
+ 261
+ 453
+ 187
+ 26
+ 198
+ 402
+ 555
+$ cat output/sim_0.out
+64000302
+38000489
+32000081
+12000582
+46000664
+36000290
+35000839
+22000258
+49000506
+75000310
+-----
+
+We produce 20 values in each bias file. Simulations of less than that
+number of values ignore the unneeded number, while simualtions of more
+than 20 will use the last bias number for all remoaining values past
+20. As an exercise, adjust the code to produce the same number of
+bias values as is needed for each simulation. As a further exercise,
+modify the script to generate a unique seed value for each simulation,
+which is a common practice in ensemble computations.
+
Modified: SwiftTutorials/swift-cray-tutorial/part06/apps
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part06/apps 2013-08-27 23:40:14 UTC (rev 7013)
+++ SwiftTutorials/swift-cray-tutorial/part06/apps 2013-08-28 00:39:20 UTC (rev 7014)
@@ -1,4 +1,4 @@
-raven simulate /home/users/p01537/swift-cray-tutorial/app/simulate.sh
-raven analyze /home/users/p01537/swift-cray-tutorial/app/stats.sh
-raven genseed /home/users/p01537/swift-cray-tutorial/app/simulate.sh
-raven genbias /home/users/p01537/swift-cray-tutorial/app/simulate.sh
+raven simulate simulate.sh
+raven stats stats.sh
+raven genseed simulate.sh
+raven genbias simulate.sh
Modified: SwiftTutorials/swift-cray-tutorial/part06/p6.swift
===================================================================
--- SwiftTutorials/swift-cray-tutorial/part06/p6.swift 2013-08-27 23:40:14 UTC (rev 7013)
+++ SwiftTutorials/swift-cray-tutorial/part06/p6.swift 2013-08-28 00:39:20 UTC (rev 7014)
@@ -19,21 +19,21 @@
"-n" sim_count stdout=@out stderr=@log;
}
-app (file out) analyze (file s[])
+app (file out, file log) analyze (file s[])
{
- analyze @filenames(s) stdout=@out;
+ stats @filenames(s) stdout=@out stderr=@log;
}
# Command line arguments
-int nsim = @toInt(@arg("nsim", "10")); # number of simulation programs to run
-int range = @toInt(@arg("range", "100")); # range of the generated random numbers
-int count = @toInt(@arg("count", "10")); # number of values generated per simulation
-int steps = @toInt(@arg("steps", "1")); # number of timesteps (seconds) per simulation
+int nsim = @toInt(@arg("nsim", "10")); # number of simulation programs to run
+int steps = @toInt(@arg("steps", "1")); # number of timesteps (seconds) per simulation
+int range = @toInt(@arg("range", "100")); # range of the generated random numbers
+int values = @toInt(@arg("values", "10")); # number of values generated per simulation
# Main script and data
-tracef("\n*** Script parameters: nsim=%i range=%i count=%i\n\n", nsim, range, count);
+tracef("\n*** Script parameters: nsim=%i range=%i num values=%i\n\n", nsim, range, values);
file seedfile<"output/seed.dat">; # Dynamically generated bias for simulation ensemble
seedfile = genseed(1);
@@ -48,9 +48,10 @@
file simout <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
file simlog <single_file_mapper; file=@strcat("output/sim_",i,".log")>;
biasfile = genbias(1000, 20);
- (simout,simlog) = simulation(steps, range, biasfile, 100000, count);
+ (simout,simlog) = simulation(steps, range, biasfile, 1000000, values);
sims[i] = simout;
}
-file stats<"output/stats.out">; # Final output file: average of all "simulations"
-stats = analyze(sims);
+file stats_out<"output/average.out">;
+file stats_log<"output/average.log">;
+(stats_out,stats_log) = analyze(sims);
Modified: SwiftTutorials/swift-cray-tutorial/setup.sh
===================================================================
--- SwiftTutorials/swift-cray-tutorial/setup.sh 2013-08-27 23:40:14 UTC (rev 7013)
+++ SwiftTutorials/swift-cray-tutorial/setup.sh 2013-08-28 00:39:20 UTC (rev 7014)
@@ -99,7 +99,6 @@
<profile namespace="karajan" key="jobThrottle">3.20</profile>
<profile namespace="karajan" key="initialScore">10000</profile>
<workdirectory>/lus/scratch/{env.USER}/swiftwork</workdirectory>
- <profile namespace="swift" key="stagingMethod">sfs</profile>
</pool>
<pool handle="ravenMED">
@@ -118,7 +117,6 @@
<profile namespace="karajan" key="jobThrottle">2.56</profile>
<profile namespace="karajan" key="initialScore">10000</profile>
<workdirectory>/lus/scratch/{env.USER}/swiftwork</workdirectory>
- <profile namespace="swift" key="stagingMethod">sfs</profile>
</pool>
<pool handle="ravenGPU">
@@ -137,7 +135,6 @@
<profile namespace="karajan" key="jobThrottle">5.00</profile>
<profile namespace="karajan" key="initialScore">10000</profile>
<workdirectory>/lus/scratch/{env.USER}/swiftwork</workdirectory>
- <profile namespace="swift" key="stagingMethod">sfs</profile>
</pool>
</config>
More information about the Swift-commit
mailing list