[Swift-commit] r6421 - in SwiftTutorials/UofC_2013-04-09: . part06 part07
wilde at ci.uchicago.edu
wilde at ci.uchicago.edu
Tue Apr 9 09:48:47 CDT 2013
Author: wilde
Date: 2013-04-09 09:48:47 -0500 (Tue, 09 Apr 2013)
New Revision: 6421
Added:
SwiftTutorials/UofC_2013-04-09/part06/
SwiftTutorials/UofC_2013-04-09/part06/README
SwiftTutorials/UofC_2013-04-09/part06/p6.swift
SwiftTutorials/UofC_2013-04-09/part06/tc
SwiftTutorials/UofC_2013-04-09/part07/
SwiftTutorials/UofC_2013-04-09/part07/README
SwiftTutorials/UofC_2013-04-09/part07/p7.swift
SwiftTutorials/UofC_2013-04-09/part07/sites.xml
SwiftTutorials/UofC_2013-04-09/part07/tc
Log:
Added new parts for arg passing and cluster execution.
Added: SwiftTutorials/UofC_2013-04-09/part06/README
===================================================================
--- SwiftTutorials/UofC_2013-04-09/part06/README (rev 0)
+++ SwiftTutorials/UofC_2013-04-09/part06/README 2013-04-09 14:48:47 UTC (rev 6421)
@@ -0,0 +1,28 @@
+
+Swift script - add arguments to the "simulation" program to control number of steps,
+ range of output, and number of output lines,
+
+
+Test new versions of the "simulation" program:
+
+ random.sh # original
+ random2.sh [runtime] # runtime in seconds; defaults to 1 sec
+ random3.sh [runtime] [range] # requests random numbers in {0:range-1}
+ random4.sh [runtime] [range] [count] # requests "count" random numbers in {0:range}
+
+p6.swift changes to random2 for the simulation program and adds an argument.
+
+Run the program:
+
+$ swift -tc.file tc p6.swift -steps 3 # each simulation takes 3 seconds
+
+Next we need to use the parallel cluster for our simulations!
+
+Try:
+
+- change to random3.sh and add the extra argument "range"
+
+- change to random4.sh and add yet another argument "count"
+
+
+
Added: SwiftTutorials/UofC_2013-04-09/part06/p6.swift
===================================================================
--- SwiftTutorials/UofC_2013-04-09/part06/p6.swift (rev 0)
+++ SwiftTutorials/UofC_2013-04-09/part06/p6.swift 2013-04-09 14:48:47 UTC (rev 6421)
@@ -0,0 +1,25 @@
+type file;
+
+app (file o) mysim2 (int timesteps)
+{
+ random2 timesteps stdout=@filename(o);
+}
+
+app (file o) analyze (file s[])
+{
+ average @filenames(s) stdout=@filename(o);
+}
+
+file sims[];
+int nsim = @toInt(@arg("nsim","10"));
+
+int steps = @toInt(@arg("steps","1"));
+
+foreach i in [0:nsim-1] {
+ file simout <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
+ simout = mysim2(steps);
+ sims[i] = simout;
+}
+
+file stats<"output/average.out">;
+stats = analyze(sims);
Added: SwiftTutorials/UofC_2013-04-09/part06/tc
===================================================================
--- SwiftTutorials/UofC_2013-04-09/part06/tc (rev 0)
+++ SwiftTutorials/UofC_2013-04-09/part06/tc 2013-04-09 14:48:47 UTC (rev 6421)
@@ -0,0 +1,2 @@
+localhost random2 random2.sh
+localhost average avg.sh
Added: SwiftTutorials/UofC_2013-04-09/part07/README
===================================================================
--- SwiftTutorials/UofC_2013-04-09/part07/README (rev 0)
+++ SwiftTutorials/UofC_2013-04-09/part07/README 2013-04-09 14:48:47 UTC (rev 6421)
@@ -0,0 +1,81 @@
+
+Swift script - execution on the parallel cluster
+
+
+Til now we've been running our "simulations" on the local login host.
+
+Now we'll run them on the parallel cluster.
+
+The script will be the same as previous, but we change the tool catalog to
+indicate that our "simulation app" (random4.sh) should be run on the
+"westmere" partition of the Midway cluster using the "slurm" scheduler:
+
+We changed file "tc" from:
+
+localhost random4 random4.sh
+localhost average avg.sh
+
+to:
+
+westmere random4 random4.sh
+localhost average avg.sh
+
+And we added the "sites.xml" file which specifies two "pools" of resources,
+the "local host" pool (which we've been using till now) and the "westmere" pool.
+
+To run, we now add the sites.xml file to the swift command line:
+
+$ swift -tc.file tc -sites.file sites.xml p7.swift -steps=10 -range=1000 -count=6 -nsim=100
+
+Note that we're now asking for 100 simulations to be run, each of which
+is to run for 10 steps (ie 10 seconds).
+
+The sites file looks like this:
+
+<config>
+
+ <pool handle="localhost">
+ <execution provider="local"/>
+ <filesystem provider="local"/>
+ <workdirectory>/scratch/midway/{env.USER}/swiftwork</workdirectory>
+ </pool>
+
+ <pool handle="westmere">
+ <execution provider="coaster" jobmanager="local:slurm"/>
+
+ <!-- Set partition and account here: -->
+ <profile namespace="globus" key="queue">westmere</profile>
+ <profile namespace="globus" key="ppn">12</profile>
+ <!-- <profile namespace="globus" key="project">pi-wilde</profile> -->
+
+ <!-- Set number of jobs and nodes per job here: -->
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="maxnodes">1</profile>
+ <profile namespace="globus" key="nodegranularity">1</profile>
+ <profile namespace="globus" key="jobsPerNode">12</profile> <!-- apps per node! -->
+ <profile namespace="karajan" key="jobThrottle">.11</profile> <!-- eg .11 -> 12 -->
+
+ <!-- Set estimated app time (maxwalltime) and requested job time (maxtime) here: -->
+ <profile namespace="globus" key="maxWalltime">00:15:00</profile>
+ <profile namespace="globus" key="maxtime">1800</profile> <!-- in seconds! -->
+
+ <!-- Set data staging model and work dir here: -->
+ <filesystem provider="local"/>
+ <workdirectory>/scratch/midway/{env.USER}/swiftwork</workdirectory>
+
+ <!-- Typically leave these constant: -->
+ <profile namespace="globus" key="slurm.exclusive">false</profile>
+ <profile namespace="globus" key="highOverAllocation">100</profile>
+ <profile namespace="globus" key="lowOverAllocation">100</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ </pool>
+
+</config>
+
+Try:
+
+- running more app() calls at once: change jobsPerNode and jobThrottle
+
+- requesting more than 1 slurm job at once: change slots
+
+- requesting nodes in units greater than 1: change maxnodes and nodeGranularity
Added: SwiftTutorials/UofC_2013-04-09/part07/p7.swift
===================================================================
--- SwiftTutorials/UofC_2013-04-09/part07/p7.swift (rev 0)
+++ SwiftTutorials/UofC_2013-04-09/part07/p7.swift 2013-04-09 14:48:47 UTC (rev 6421)
@@ -0,0 +1,27 @@
+type file;
+
+app (file o) mysim4 (int timesteps, int range, int count)
+{
+ random4 timesteps range count stdout=@filename(o);
+}
+
+app (file o) analyze (file s[])
+{
+ average @filenames(s) stdout=@filename(o);
+}
+
+file sims[];
+int nsim = @toInt(@arg("nsim", "10"));
+
+int steps = @toInt(@arg("steps", "1"));
+int range = @toInt(@arg("range", "100"));
+int count = @toInt(@arg("count", "10"));
+
+foreach i in [0:nsim-1] {
+ file simout <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
+ simout = mysim4(steps,range,count);
+ sims[i] = simout;
+}
+
+file stats<"output/average.out">;
+stats = analyze(sims);
Added: SwiftTutorials/UofC_2013-04-09/part07/sites.xml
===================================================================
--- SwiftTutorials/UofC_2013-04-09/part07/sites.xml (rev 0)
+++ SwiftTutorials/UofC_2013-04-09/part07/sites.xml 2013-04-09 14:48:47 UTC (rev 6421)
@@ -0,0 +1,39 @@
+<config>
+
+ <pool handle="localhost">
+ <execution provider="local"/>
+ <filesystem provider="local"/>
+ <workdirectory>/scratch/midway/{env.USER}/swiftwork</workdirectory>
+ </pool>
+
+ <pool handle="westmere">
+ <execution provider="coaster" jobmanager="local:slurm"/>
+
+ <!-- Set partition and account here: -->
+ <profile namespace="globus" key="queue">westmere</profile>
+ <profile namespace="globus" key="ppn">12</profile>
+ <!-- <profile namespace="globus" key="project">pi-wilde</profile> -->
+
+ <!-- Set number of jobs and nodes per job here: -->
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="maxnodes">1</profile>
+ <profile namespace="globus" key="nodegranularity">1</profile>
+ <profile namespace="globus" key="jobsPerNode">12</profile> <!-- apps per node! -->
+ <profile namespace="karajan" key="jobThrottle">.11</profile> <!-- eg .11 -> 12 -->
+
+ <!-- Set estimated app time (maxwalltime) and requested job time (maxtime) here: -->
+ <profile namespace="globus" key="maxWalltime">00:15:00</profile>
+ <profile namespace="globus" key="maxtime">1800</profile> <!-- in seconds! -->
+
+ <!-- Set data staging model and work dir here: -->
+ <filesystem provider="local"/>
+ <workdirectory>/scratch/midway/{env.USER}/swiftwork</workdirectory>
+
+ <!-- Typically leave these constant: -->
+ <profile namespace="globus" key="slurm.exclusive">false</profile>
+ <profile namespace="globus" key="highOverAllocation">100</profile>
+ <profile namespace="globus" key="lowOverAllocation">100</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ </pool>
+
+</config>
Added: SwiftTutorials/UofC_2013-04-09/part07/tc
===================================================================
--- SwiftTutorials/UofC_2013-04-09/part07/tc (rev 0)
+++ SwiftTutorials/UofC_2013-04-09/part07/tc 2013-04-09 14:48:47 UTC (rev 6421)
@@ -0,0 +1,2 @@
+westmere random4 random4.sh
+localhost average avg.sh
More information about the Swift-commit
mailing list