[Swift-commit] r7414 - SwiftApps/tryswift/scripts
davidk at ci.uchicago.edu
davidk at ci.uchicago.edu
Wed Dec 11 16:22:45 CST 2013
Author: davidk
Date: 2013-12-11 16:22:44 -0600 (Wed, 11 Dec 2013)
New Revision: 7414
Added:
SwiftApps/tryswift/scripts/004-multipleapps.html
SwiftApps/tryswift/scripts/004-multipleapps.swift
SwiftApps/tryswift/scripts/004-multipleapps.txt
SwiftApps/tryswift/scripts/005-multistage.html
SwiftApps/tryswift/scripts/005-multistage.png
SwiftApps/tryswift/scripts/005-multistage.swift
SwiftApps/tryswift/scripts/005-multistage.txt
Removed:
SwiftApps/tryswift/scripts/004-analysis.html
SwiftApps/tryswift/scripts/004-analysis.swift
SwiftApps/tryswift/scripts/004-analysis.txt
SwiftApps/tryswift/scripts/activeplot.png
SwiftApps/tryswift/scripts/cumulativeplot.png
SwiftApps/tryswift/scripts/p4.html
SwiftApps/tryswift/scripts/p4.swift
SwiftApps/tryswift/scripts/p4.txt
SwiftApps/tryswift/scripts/p5.html
SwiftApps/tryswift/scripts/p5.swift
SwiftApps/tryswift/scripts/p5.txt
SwiftApps/tryswift/scripts/p6.html
SwiftApps/tryswift/scripts/p6.swift
SwiftApps/tryswift/scripts/p6.txt
SwiftApps/tryswift/scripts/part04.png
SwiftApps/tryswift/scripts/part05.png
SwiftApps/tryswift/scripts/part06.png
SwiftApps/tryswift/scripts/part4.txt
SwiftApps/tryswift/scripts/part5.txt
SwiftApps/tryswift/scripts/part6.txt
Modified:
SwiftApps/tryswift/scripts/002-helloworld.swift
SwiftApps/tryswift/scripts/003-foreach.html
SwiftApps/tryswift/scripts/003-foreach.swift
Log:
Remove unused script
New 005-multistage example based on tutorial scripts
Some doc improvements and script improvements based on feedback
Modified: SwiftApps/tryswift/scripts/002-helloworld.swift
===================================================================
--- SwiftApps/tryswift/scripts/002-helloworld.swift 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/002-helloworld.swift 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,9 +1,9 @@
type file;
-app (file out) echo (string s)
+app (file out) echo_app (string s)
{
echo s stdout=filename(out);
}
file out <"out.txt">;
-out = echo("Hello world!");
+out = echo_app("Hello world!");
Modified: SwiftApps/tryswift/scripts/003-foreach.html
===================================================================
--- SwiftApps/tryswift/scripts/003-foreach.html 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/003-foreach.html 2013-12-11 22:22:44 UTC (rev 7414)
@@ -9,8 +9,8 @@
In this example, we first change our application. Instead of using "echo",
we use an app called simulate. The simulate application serves as a trivial
-proxy for any more complex scientific simulation application. It generates
-and prints a set of one or more random integers
+proxy for any more complex scientific simulation application. In this
+example, simulate will print a single number in the range of 1-100.
<pre>
app (file o) simulate ()
Modified: SwiftApps/tryswift/scripts/003-foreach.swift
===================================================================
--- SwiftApps/tryswift/scripts/003-foreach.swift 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/003-foreach.swift 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,6 +1,6 @@
type file;
-app (file o) simulate ()
+app (file o) simulate_app ()
{
simulate stdout=filename(o);
}
@@ -8,5 +8,5 @@
foreach i in [1:10] {
string fname=strcat("output/sim_", i, ".out");
file f <single_file_mapper; file=fname>;
- f = simulate();
+ f = simulate_app();
}
Deleted: SwiftApps/tryswift/scripts/004-analysis.html
===================================================================
--- SwiftApps/tryswift/scripts/004-analysis.html 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/004-analysis.html 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,69 +0,0 @@
-<html>
-
-<head>
-<link href="formatting.css" rel="stylesheet" media="screen">
-</head>
-
-<body>
-<h2>Multiple apps</h2>
-
-<p>After all the parallel simulations in an ensemble run have completed,
-it is typically necessary to gather and analyze their results with some
-kind of post-processing analysis program or script. This script shows
-an example of this.</p>
-
-<p>The first change in this script is to the simulate script:</p>
-
-<pre>
-app (file o) simulate (int time)
-{
- simulate "-t" time stdout=filename(o);
-}
-</pre>
-
-<p>Simulate now takes an argument, time. The command "simulate -t 10" will sleep
-for 10 seconds before printing a value. This is an example of how to pass command
-line arguments to an app in Swift.</p>
-
-<p>We introduce a new app call called stats:</p>
-
-<pre>
-app (file o) stats (file s[])
-{
- stats filenames(s) stdout=filename(o);
-}
-</pre>
-
-<p>The stats app function takes an array of files as input (file s[]). The stats app takes a list of
-files, reads the numbers contained inside, and prints the average value. The filenames() function simply
-prints a list of all filenames contained within a file array.</p>
-
-<p>Within the foreach loop:
-<pre>
- simout = simulate(time);
- sims[i] = simout;
-</pre>
-
-<p>We now add each simout file to the sims array before finally passing all the files to stats</p>
-<pre>
-foreach i in [1:nsims] {
- string fname = strcat("output/sim_",i,".out");
- file simout <single_file_mapper; file=fname>;
- simout = simulate(time);
- sims[i] = simout;
-}
-
-file average<"output/average.out">;
-average = stats(sims);
-</pre>
-
-Execute the script and view output/average.out to verify it succeeded.
-
-<h3>Exercises</h3>
-<ul>
-<li>Simulate takes another command line option, -r. The -r option sets the range of random numbers it generates. Call simulate with the added options -r 1000.</li>
-<li>Modify the simulate app so that it takes a second int representing range (hint: multiple inputs are separated by commas).</li>
-</ul>
-
-</body>
-</html>
Deleted: SwiftApps/tryswift/scripts/004-analysis.swift
===================================================================
--- SwiftApps/tryswift/scripts/004-analysis.swift 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/004-analysis.swift 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,25 +0,0 @@
-type file;
-
-app (file o) simulate (int time)
-{
- simulate "-t" time stdout=filename(o);
-}
-
-app (file o) stats (file s[])
-{
- stats filenames(s) stdout=filename(o);
-}
-
-file sims[];
-int time = 5;
-int nsims = 10;
-
-foreach i in [1:nsims] {
- string fname = strcat("output/sim_",i,".out");
- file simout <single_file_mapper; file=fname>;
- simout = simulate(time);
- sims[i] = simout;
-}
-
-file average<"output/average.out">;
-average = stats(sims);
Deleted: SwiftApps/tryswift/scripts/004-analysis.txt
===================================================================
--- SwiftApps/tryswift/scripts/004-analysis.txt 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/004-analysis.txt 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1 +0,0 @@
-Multiple apps
Copied: SwiftApps/tryswift/scripts/004-multipleapps.html (from rev 7151, SwiftApps/tryswift/scripts/004-analysis.html)
===================================================================
--- SwiftApps/tryswift/scripts/004-multipleapps.html (rev 0)
+++ SwiftApps/tryswift/scripts/004-multipleapps.html 2013-12-11 22:22:44 UTC (rev 7414)
@@ -0,0 +1,72 @@
+<html>
+
+<head>
+<link href="formatting.css" rel="stylesheet" media="screen">
+</head>
+
+<body>
+<h2>Multiple apps</h2>
+
+<p>After all the parallel simulations in an ensemble run have completed,
+it is typically necessary to gather and analyze their results with some
+kind of post-processing analysis program or script. This script shows
+an example of this.</p>
+
+<p>The first change in this script is to the simulate script:</p>
+
+<pre>
+app (file o) simulate (int time)
+{
+ simulate "-t" time stdout=filename(o);
+}
+</pre>
+
+<p>Simulate now takes an argument, time. The command "simulate -t 10" will sleep
+for 10 seconds before printing a value. This is an example of how to pass command
+line arguments to an app in Swift.</p>
+
+<p>We introduce a new app call called stats:</p>
+
+<pre>
+app (file o) stats (file s[])
+{
+ stats filenames(s) stdout=filename(o);
+}
+</pre>
+
+<p>The stats app function takes an array of files as input (file s[]). The stats app takes a list of
+files, reads the numbers contained inside, and prints the average value. The filenames() function simply
+prints a list of all filenames contained within a file array.</p>
+
+<p>Within the foreach loop:
+<pre>
+ simout = simulate(time);
+ sims[i] = simout;
+</pre>
+
+<p>We now add each simout file to the sims array before finally passing all the files to stats</p>
+<pre>
+foreach i in [1:nsims] {
+ string fname = strcat("output/sim_",i,".out");
+ file simout <single_file_mapper; file=fname>;
+ simout = simulate(time);
+ sims[i] = simout;
+}
+
+file average<"output/average.out">;
+average = stats(sims);
+</pre>
+
+Execute the script and view output/average.out to verify it succeeded.
+
+<h3>Exercises</h3>
+<ul>
+<li>Modify the simulate_app function so that it accepts a second int that will
+ representing range (hint: multiple inputs are separated by commas).</li>
+<li>Modify the simulate command line arguments. The current arguments are "-t time".
+ Simulate takes another command line option, -r. The -r option sets the range of
+ random numbers it generates. Call simulate with the added options -r 1000.</li>
+</ul>
+
+</body>
+</html>
Copied: SwiftApps/tryswift/scripts/004-multipleapps.swift (from rev 7151, SwiftApps/tryswift/scripts/004-analysis.swift)
===================================================================
--- SwiftApps/tryswift/scripts/004-multipleapps.swift (rev 0)
+++ SwiftApps/tryswift/scripts/004-multipleapps.swift 2013-12-11 22:22:44 UTC (rev 7414)
@@ -0,0 +1,25 @@
+type file;
+
+app (file o) simulate_app (int time)
+{
+ simulate "-t" time stdout=filename(o);
+}
+
+app (file o) stats_app (file s[])
+{
+ stats filenames(s) stdout=filename(o);
+}
+
+file sims[];
+int time = 5;
+int nsims = 10;
+
+foreach i in [1:nsims] {
+ string fname = strcat("output/sim_",i,".out");
+ file simout <single_file_mapper; file=fname>;
+ simout = simulate_app(time);
+ sims[i] = simout;
+}
+
+file average <"output/average.out">;
+average = stats_app(sims);
Copied: SwiftApps/tryswift/scripts/004-multipleapps.txt (from rev 7151, SwiftApps/tryswift/scripts/004-analysis.txt)
===================================================================
--- SwiftApps/tryswift/scripts/004-multipleapps.txt (rev 0)
+++ SwiftApps/tryswift/scripts/004-multipleapps.txt 2013-12-11 22:22:44 UTC (rev 7414)
@@ -0,0 +1 @@
+Multiple apps
Added: SwiftApps/tryswift/scripts/005-multistage.html
===================================================================
--- SwiftApps/tryswift/scripts/005-multistage.html (rev 0)
+++ SwiftApps/tryswift/scripts/005-multistage.html 2013-12-11 22:22:44 UTC (rev 7414)
@@ -0,0 +1,42 @@
+<html>
+
+<head>
+<link href="formatting.css" rel="stylesheet" media="screen">
+</head>
+
+<body>
+<h2>Multi-stage workflows</h2>
+
+<p>This example expands the workflow pattern of the previous example by adding additional stages to the workflow.
+Here, we generate a dynamic seed value that will be used by all of the simulations, and for each simulation,
+we run an pre-processing application to generate a unique "bias file". This pattern is shown below, followed
+by the Swift script.</p>
+
+<img src="005-multistage.png"/>
+
+<p>Note that the workflow is based on data flow dependencies: each simulation depends on the seed value, calculated
+in these two dependent statements:</p>
+
+<pre>
+seedfile = genseed_app(1);
+int seedval = readData(seedfile);
+</pre>
+
+The workflow also depends on the bias file, computed and then consumed in these two dependent statements:
+<pre>
+biasfile = genbias_app(1000, 20);
+(simout,simlog) = simulation_app(steps, range, biasfile, 1000000, values);
+</pre>
+
+We produce 20 values in each bias file. Simulations of less than 20 values
+ignore the unneeded numbers, while simualtions of more than 20 will use the
+last bias number for all values past 20.
+
+<h3>Exercises</h3>
+<ul>
+<li>Adjust the code to produce the same number of bias values as is needed for each simulation.</li>
+<li>Modify the script to generate a unique seed value for each simulation</li>
+</ul>
+
+</body>
+</html>
Added: SwiftApps/tryswift/scripts/005-multistage.png
===================================================================
(Binary files differ)
Property changes on: SwiftApps/tryswift/scripts/005-multistage.png
___________________________________________________________________
Added: svn:mime-type
+ application/octet-stream
Added: SwiftApps/tryswift/scripts/005-multistage.swift
===================================================================
--- SwiftApps/tryswift/scripts/005-multistage.swift (rev 0)
+++ SwiftApps/tryswift/scripts/005-multistage.swift 2013-12-11 22:22:44 UTC (rev 7414)
@@ -0,0 +1,54 @@
+type file;
+
+app (file out) genseed_app (int nseeds)
+{
+ genseed "-r" 2000000 "-n" nseeds stdout=@out;
+}
+
+app (file out) genbias_app (int bias_range, int nvalues)
+{
+ genbias "-r" bias_range "-n" nvalues stdout=@out;
+}
+
+app (file out, file log) simulation_app (int timesteps, int sim_range,
+ file bias_file, int scale, int sim_count)
+{
+ simulate "-t" timesteps "-r" sim_range "-B" @bias_file "-x" scale
+ "-n" sim_count stdout=@out stderr=@log;
+}
+
+app (file out, file log) analyze_app (file s[])
+{
+ stats filenames(s) stdout=@out stderr=@log;
+}
+
+# Values that shape the run
+int nsim = 10; # number of simulation programs to run
+int steps = 1; # number of timesteps (seconds) per simulation
+int range = 100; # range of the generated random numbers
+int values = 10; # number of values generated per simulation
+
+# Main script and data
+tracef("\n*** Script parameters: nsim=%i range=%i num values=%i\n\n", nsim, range, values);
+
+# Dynamically generated bias for simulation ensemble
+file seedfile<"output/seed.dat">;
+seedfile = genseed_app(1);
+
+int seedval = readData(seedfile);
+tracef("Generated seed=%i\n", seedval);
+
+file sims[]; # Array of files to hold each simulation output
+
+foreach i in [0:nsim-1] {
+ file biasfile <single_file_mapper; file=strcat("output/bias_",i,".dat")>;
+ file simout <single_file_mapper; file=strcat("output/sim_",i,".out")>;
+ file simlog <single_file_mapper; file=strcat("output/sim_",i,".log")>;
+ biasfile = genbias_app(1000, 20);
+ (simout,simlog) = simulation_app(steps, range, biasfile, 1000000, values);
+ sims[i] = simout;
+}
+
+file stats_out<"output/average.out">;
+file stats_log<"output/average.log">;
+(stats_out,stats_log) = analyze_app(sims);
Added: SwiftApps/tryswift/scripts/005-multistage.txt
===================================================================
--- SwiftApps/tryswift/scripts/005-multistage.txt (rev 0)
+++ SwiftApps/tryswift/scripts/005-multistage.txt 2013-12-11 22:22:44 UTC (rev 7414)
@@ -0,0 +1 @@
+Multi-stage workflows
Deleted: SwiftApps/tryswift/scripts/activeplot.png
===================================================================
(Binary files differ)
Deleted: SwiftApps/tryswift/scripts/cumulativeplot.png
===================================================================
(Binary files differ)
Deleted: SwiftApps/tryswift/scripts/p4.html
===================================================================
--- SwiftApps/tryswift/scripts/p4.html 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/p4.html 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,816 +0,0 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
- "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
-<head>
-<meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
-<meta name="generator" content="AsciiDoc 8.6.4" />
-<title>Part 4: Running a parallel ensemble on Cray compute nodes</title>
-<style type="text/css">
-/* Sans-serif font. */
-h1, h2, h3, h4, h5, h6,
-div.title, caption.title,
-thead, p.table.header,
-div#toctitle,
-span#author, span#revnumber, span#revdate, span#revremark,
-div#footer {
- font-family: Arial,Helvetica,sans-serif;
-}
-
-/* Serif font. */
-div.sectionbody {
- font-family: Georgia,"Times New Roman",Times,serif;
-}
-
-/* Monospace font. */
-tt {
- font-size: inherit;
-}
-
-body {
- margin: 1em 5% 1em 5%;
-}
-
-a {
- color: blue;
- text-decoration: underline;
-}
-a:visited {
- color: fuchsia;
-}
-
-em {
- font-style: italic;
- color: navy;
-}
-
-strong {
- font-weight: bold;
- color: #083194;
-}
-
-tt {
- font-size: inherit;
- color: navy;
-}
-
-h1, h2, h3, h4, h5, h6 {
- color: #527bbd;
- margin-top: 1.2em;
- margin-bottom: 0.5em;
- line-height: 1.3;
-}
-
-h1, h2, h3 {
- border-bottom: 2px solid silver;
-}
-h2 {
- padding-top: 0.5em;
-}
-h3 {
- float: left;
-}
-h3 + * {
- clear: left;
-}
-
-div.sectionbody {
- margin-left: 0;
-}
-
-hr {
- border: 1px solid silver;
-}
-
-p {
- margin-top: 0.5em;
- margin-bottom: 0.5em;
-}
-
-ul, ol, li > p {
- margin-top: 0;
-}
-ul > li { color: #aaa; }
-ul > li > * { color: black; }
-
-pre {
- padding: 0;
- margin: 0;
-}
-
-span#author {
- color: #527bbd;
- font-weight: bold;
- font-size: 1.1em;
-}
-span#email {
-}
-span#revnumber, span#revdate, span#revremark {
-}
-
-div#footer {
- font-size: small;
- border-top: 2px solid silver;
- padding-top: 0.5em;
- margin-top: 4.0em;
-}
-div#footer-text {
- float: left;
- padding-bottom: 0.5em;
-}
-div#footer-badges {
- float: right;
- padding-bottom: 0.5em;
-}
-
-div#preamble {
- margin-top: 1.5em;
- margin-bottom: 1.5em;
-}
-div.tableblock, div.imageblock, div.exampleblock, div.verseblock,
-div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
-div.admonitionblock {
- margin-top: 1.0em;
- margin-bottom: 1.5em;
-}
-div.admonitionblock {
- margin-top: 2.0em;
- margin-bottom: 2.0em;
- margin-right: 10%;
- color: #606060;
-}
-
-div.content { /* Block element content. */
- padding: 0;
-}
-
-/* Block element titles. */
-div.title, caption.title {
- color: #527bbd;
- font-weight: bold;
- text-align: left;
- margin-top: 1.0em;
- margin-bottom: 0.5em;
-}
-div.title + * {
- margin-top: 0;
-}
-
-td div.title:first-child {
- margin-top: 0.0em;
-}
-div.content div.title:first-child {
- margin-top: 0.0em;
-}
-div.content + div.title {
- margin-top: 0.0em;
-}
-
-div.sidebarblock > div.content {
- background: #ffffee;
- border: 1px solid #dddddd;
- border-left: 4px solid #f0f0f0;
- padding: 0.5em;
-}
-
-div.listingblock > div.content {
- border: 1px solid #dddddd;
- border-left: 5px solid #f0f0f0;
- background: #f8f8f8;
- padding: 0.5em;
-}
-
-div.quoteblock, div.verseblock {
- padding-left: 1.0em;
- margin-left: 1.0em;
- margin-right: 10%;
- border-left: 5px solid #f0f0f0;
- color: #777777;
-}
-
-div.quoteblock > div.attribution {
- padding-top: 0.5em;
- text-align: right;
-}
-
-div.verseblock > pre.content {
- font-family: inherit;
- font-size: inherit;
-}
-div.verseblock > div.attribution {
- padding-top: 0.75em;
- text-align: left;
-}
-/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
-div.verseblock + div.attribution {
- text-align: left;
-}
-
-div.admonitionblock .icon {
- vertical-align: top;
- font-size: 1.1em;
- font-weight: bold;
- text-decoration: underline;
- color: #527bbd;
- padding-right: 0.5em;
-}
-div.admonitionblock td.content {
- padding-left: 0.5em;
- border-left: 3px solid #dddddd;
-}
-
-div.exampleblock > div.content {
- border-left: 3px solid #dddddd;
- padding-left: 0.5em;
-}
-
-div.imageblock div.content { padding-left: 0; }
-span.image img { border-style: none; }
-a.image:visited { color: white; }
-
-dl {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-dt {
- margin-top: 0.5em;
- margin-bottom: 0;
- font-style: normal;
- color: navy;
-}
-dd > *:first-child {
- margin-top: 0.1em;
-}
-
-ul, ol {
- list-style-position: outside;
-}
-ol.arabic {
- list-style-type: decimal;
-}
-ol.loweralpha {
- list-style-type: lower-alpha;
-}
-ol.upperalpha {
- list-style-type: upper-alpha;
-}
-ol.lowerroman {
- list-style-type: lower-roman;
-}
-ol.upperroman {
- list-style-type: upper-roman;
-}
-
-div.compact ul, div.compact ol,
-div.compact p, div.compact p,
-div.compact div, div.compact div {
- margin-top: 0.1em;
- margin-bottom: 0.1em;
-}
-
-div.tableblock > table {
- border: 3px solid #527bbd;
-}
-thead, p.table.header {
- font-weight: bold;
- color: #527bbd;
-}
-tfoot {
- font-weight: bold;
-}
-td > div.verse {
- white-space: pre;
-}
-p.table {
- margin-top: 0;
-}
-/* Because the table frame attribute is overriden by CSS in most browsers. */
-div.tableblock > table[frame="void"] {
- border-style: none;
-}
-div.tableblock > table[frame="hsides"] {
- border-left-style: none;
- border-right-style: none;
-}
-div.tableblock > table[frame="vsides"] {
- border-top-style: none;
- border-bottom-style: none;
-}
-
-
-div.hdlist {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-div.hdlist tr {
- padding-bottom: 15px;
-}
-dt.hdlist1.strong, td.hdlist1.strong {
- font-weight: bold;
-}
-td.hdlist1 {
- vertical-align: top;
- font-style: normal;
- padding-right: 0.8em;
- color: navy;
-}
-td.hdlist2 {
- vertical-align: top;
-}
-div.hdlist.compact tr {
- margin: 0;
- padding-bottom: 0;
-}
-
-.comment {
- background: yellow;
-}
-
-.footnote, .footnoteref {
- font-size: 0.8em;
-}
-
-span.footnote, span.footnoteref {
- vertical-align: super;
-}
-
-#footnotes {
- margin: 20px 0 20px 0;
- padding: 7px 0 0 0;
-}
-
-#footnotes div.footnote {
- margin: 0 0 5px 0;
-}
-
-#footnotes hr {
- border: none;
- border-top: 1px solid silver;
- height: 1px;
- text-align: left;
- margin-left: 0;
- width: 20%;
- min-width: 100px;
-}
-
-div.colist td {
- padding-right: 0.5em;
- padding-bottom: 0.3em;
- vertical-align: top;
-}
-div.colist td img {
- margin-top: 0.3em;
-}
-
- at media print {
- div#footer-badges { display: none; }
-}
-
-div#toc {
- margin-bottom: 2.5em;
-}
-
-div#toctitle {
- color: #527bbd;
- font-size: 1.1em;
- font-weight: bold;
- margin-top: 1.0em;
- margin-bottom: 0.1em;
-}
-
-div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
- margin-top: 0;
- margin-bottom: 0;
-}
-div.toclevel2 {
- margin-left: 2em;
- font-size: 0.9em;
-}
-div.toclevel3 {
- margin-left: 4em;
- font-size: 0.9em;
-}
-div.toclevel4 {
- margin-left: 6em;
- font-size: 0.9em;
-}
-
-span.aqua { color: aqua; }
-span.black { color: black; }
-span.blue { color: blue; }
-span.fuchsia { color: fuchsia; }
-span.gray { color: gray; }
-span.green { color: green; }
-span.lime { color: lime; }
-span.maroon { color: maroon; }
-span.navy { color: navy; }
-span.olive { color: olive; }
-span.purple { color: purple; }
-span.red { color: red; }
-span.silver { color: silver; }
-span.teal { color: teal; }
-span.white { color: white; }
-span.yellow { color: yellow; }
-
-span.aqua-background { background: aqua; }
-span.black-background { background: black; }
-span.blue-background { background: blue; }
-span.fuchsia-background { background: fuchsia; }
-span.gray-background { background: gray; }
-span.green-background { background: green; }
-span.lime-background { background: lime; }
-span.maroon-background { background: maroon; }
-span.navy-background { background: navy; }
-span.olive-background { background: olive; }
-span.purple-background { background: purple; }
-span.red-background { background: red; }
-span.silver-background { background: silver; }
-span.teal-background { background: teal; }
-span.white-background { background: white; }
-span.yellow-background { background: yellow; }
-
-span.big { font-size: 2em; }
-span.small { font-size: 0.6em; }
-a:link { color:navy; }
-a:visited { color:navy; }
-
-.monospaced, code, pre {
- font-family: "Courier New", Courier, monospace;
- font-size: medium; /* inherit; */
- color: black;
- padding: 0;
- margin: 0;
-}
-
-/*
- background: #f8f8f8;
- border: 1px solid #dddddd;
- border-left: 5px solid #f0f0f0;
- margin-right: 10%;
-*/
-
-div.listingblock > div.content {
- padding: 0.5em;
- background: none;
- border: none;
- border-left: none;
- margin-right: none;
-}
-
-</style>
-<script type="text/javascript">
-/*<+'])');
- // Function that scans the DOM tree for header elements (the DOM2
- // nodeIterator API would be a better technique but not supported by all
- // browsers).
- var iterate = function (el) {
- for (var i = el.firstChild; i != null; i = i.nextSibling) {
- if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
- var mo = re.exec(i.tagName);
- if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
- result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
- }
- iterate(i);
- }
- }
- }
- iterate(el);
- return result;
- }
-
- var toc = document.getElementById("toc");
- var entries = tocEntries(document.getElementById("content"), toclevels);
- for (var i = 0; i < entries.length; ++i) {
- var entry = entries[i];
- if (entry.element.id == "")
- entry.element.id = "_toc_" + i;
- var a = document.createElement("a");
- a.href = "#" + entry.element.id;
- a.appendChild(document.createTextNode(entry.text));
- var div = document.createElement("div");
- div.appendChild(a);
- div.className = "toclevel" + entry.toclevel;
- toc.appendChild(div);
- }
- if (entries.length == 0)
- toc.parentNode.removeChild(toc);
-},
-
-
-/////////////////////////////////////////////////////////////////////
-// Footnotes generator
-/////////////////////////////////////////////////////////////////////
-
-/* Based on footnote generation code from:
- * http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
- */
-
-footnotes: function () {
- var cont = document.getElementById("content");
- var noteholder = document.getElementById("footnotes");
- var spans = cont.getElementsByTagName("span");
- var refs = {};
- var n = 0;
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnote") {
- n++;
- // Use [\s\S] in place of . so multi-line matches work.
- // Because JavaScript has no s (dotall) regex flag.
- note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
- noteholder.innerHTML +=
- "<div class='footnote' id='_footnote_" + n + "'>" +
- "<a href='#_footnoteref_" + n + "' title='Return to text'>" +
- n + "</a>. " + note + "</div>";
- spans[i].innerHTML =
- "[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- var id =spans[i].getAttribute("id");
- if (id != null) refs["#"+id] = n;
- }
- }
- if (n == 0)
- noteholder.parentNode.removeChild(noteholder);
- else {
- // Process footnoterefs.
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnoteref") {
- var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
- href = href.match(/#.*/)[0]; // Because IE return full URL.
- n = refs[href];
- spans[i].innerHTML =
- "[<a href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- }
- }
- }
-}
-
-}
-/*]]>*/
-</script>
-</head>
-<body class="article" style="max-width:800px">
-<div id="header">
-<h1>Part 4: Running a parallel ensemble on Cray compute nodes</h1>
-</div>
-<div id="content">
-<div id="preamble">
-<div class="sectionbody">
-<div class="paragraph"><p><tt>p4.swift</tt> will run our mock "simulation"
-applications on Cray compute nodes. The script is similar to as
-<tt>p3.swift</tt>, but specifies that each simulation app invocation should
-additionally return the log file which the application writes to
-<tt>stderr</tt>.</p></div>
-<div class="paragraph"><p>Now when you run <tt>swift p4.swift</tt> you’ll see that two types output
-files will placed in the <tt>output/</tt> directory: <tt>sim_N.out</tt> and
-<tt>sim_N.log</tt>. The log files provide data on the runtime environment of
-each app invocation. For example:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ cat output/sim_0.log
-Called as: /home/users/p01532/swift-cray-tutorial/app/simulate.sh: --timesteps 1 --range 100 --nvalues 5
-
-Start time: Tue Aug 27 12:17:43 CDT 2013
-Running on node: nid00018
-Running as user: uid=61532(p01532) gid=61532 groups=61532
-
-Simulation parameters:
-
-bias=0
-biasfile=none
-initseed=none
-log=yes
-paramfile=none
-range=100
-scale=1
-seedfile=none
-timesteps=1
-output width=8
-
-Environment:
-
-ALPS_APP_DEPTH=32
-ASSEMBLER_X86_64=/opt/cray/cce/8.2.0.173/cray-binutils/x86_64-unknown-linux-gnu/bin/as
-ASYNCPE_DIR=/opt/cray/xt-asyncpe/5.23.02
-ASYNCPE_VERSION=5.23.02
-...</tt></pre>
-</div></div>
-<div class="paragraph"><p>To tell Swift to run the apps on compute nodes, we specify in the
-<tt>apps</tt> file that the apps should be executed on the <tt>raven</tt> site
-(instead of the <tt>localhost</tt> site). We can specify the location of
-each app in the third field of the <tt>apps</tt> file, with either an
-absolute pathname or the name of an executable to be located in
-<tt>PATH</tt>). Here we use the latter form:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ cat apps
-raven simulate simulate.sh
-raven stats stats.sh</tt></pre>
-</div></div>
-<div class="paragraph"><p>You can experiment, for example, with an alternate version of stats.sh by specfying that app’s location explicitly:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ cat apps
-raven simulate simulate.sh
-raven stats /home/users/p01532/bin/my-alt-stats.sh</tt></pre>
-</div></div>
-<div class="paragraph"><p>We can see that when we run many apps requesting a larger set of nodes (6), we are indeed running on the compute nodes:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ swift p4.swift -nsim=1000 -steps=1
-Swift 0.94.1 RC2 swift-r6895 cog-r3765
-
-RunID: 20130827-1638-t23ax37a
-Progress: time: Tue, 27 Aug 2013 16:38:11 -0500
-Progress: time: Tue, 27 Aug 2013 16:38:12 -0500 Initializing:966
-Progress: time: Tue, 27 Aug 2013 16:38:13 -0500 Selecting site:499 Submitting:500 Submitted:1
-Progress: time: Tue, 27 Aug 2013 16:38:14 -0500 Selecting site:499 Stage in:1 Submitted:500
-Progress: time: Tue, 27 Aug 2013 16:38:16 -0500 Selecting site:499 Submitted:405 Active:95 Stage out:1
-Progress: time: Tue, 27 Aug 2013 16:38:17 -0500 Selecting site:430 Submitted:434 Active:66 Stage out:1 Finished successfully:69
-Progress: time: Tue, 27 Aug 2013 16:38:18 -0500 Selecting site:388 Submitted:405 Active:95 Stage out:1 Finished successfully:111
-...
-Progress: time: Tue, 27 Aug 2013 16:38:30 -0500 Stage in:1 Submitted:93 Active:94 Finished successfully:812
-Progress: time: Tue, 27 Aug 2013 16:38:31 -0500 Submitted:55 Active:95 Stage out:1 Finished successfully:849
-Progress: time: Tue, 27 Aug 2013 16:38:32 -0500 Active:78 Stage out:1 Finished successfully:921
-Progress: time: Tue, 27 Aug 2013 16:38:34 -0500 Active:70 Stage out:1 Finished successfully:929
-Progress: time: Tue, 27 Aug 2013 16:38:37 -0500 Stage in:1 Finished successfully:1000
-Progress: time: Tue, 27 Aug 2013 16:38:38 -0500 Stage out:1 Finished successfully:1000
-Final status: Tue, 27 Aug 2013 16:38:38 -0500 Finished successfully:1001
-
-$ grep "on node:" output/*log | head
-output/sim_0.log:Running on node: nid00063
-output/sim_100.log:Running on node: nid00060
-output/sim_101.log:Running on node: nid00061
-output/sim_102.log:Running on node: nid00032
-output/sim_103.log:Running on node: nid00060
-output/sim_104.log:Running on node: nid00061
-output/sim_105.log:Running on node: nid00032
-output/sim_106.log:Running on node: nid00060
-output/sim_107.log:Running on node: nid00061
-output/sim_108.log:Running on node: nid00062
-
-$ grep "on node:" output/*log | awk '{print $4}' | sort | uniq -c
- 158 nid00032
- 156 nid00033
- 171 nid00060
- 178 nid00061
- 166 nid00062
- 171 nid00063
-$ hostname
-raven
-$ hostname -f
-nid00008</tt></pre>
-</div></div>
-<div class="paragraph"><p>Swift’s <tt>sites.xml</tt> configuration file allows many parameters to
-specify how jobs should be run on a given cluster. Consider for
-example that Raven has several queues, each with limitiations on the
-size of jobs that can be run in them. All Raven queues will only run
-2 jobs per user at one. The Raven queue "small" will only allow up to
-4 nodes per job and 1 hours of walltime per job. The following
-site.xml parameters will allow us to match this:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt> <profile namespace="globus" key="queue">small</profile>
- <profile namespace="globus" key="slots">2</profile>
- <profile namespace="globus" key="maxNodes">4</profile>
- <profile namespace="globus" key="nodeGranularity">4</profile></tt></pre>
-</div></div>
-<div class="paragraph"><p>To run large jobs, we can specify:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt> <profile namespace="globus" key="slots">2</profile>
- <profile namespace="globus" key="maxNodes">8</profile>
- <profile namespace="globus" key="nodeGranularity">8</profile>
- <profile namespace="karajan" key="jobThrottle">50.0</profile>
- <profile namespace="globus" key="maxTime">21600</profile>
- <profile namespace="globus" key="lowOverAllocation">10000</profile>
- <profile namespace="globus" key="highOverAllocation">10000</profile></tt></pre>
-</div></div>
-<div class="paragraph"><p>This will enable 512 Swift apps (2 x 8 x 32) to run concurrently
-within 2 8-node jobs on Raven’s 32-core nodes. It results in the
-following two PBS jobs submitted by Swift to "provision" compute nodes
-to run thousands of apps, 512 at a time:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ qstat -u $USER
-
-Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
---------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
-288637.sdb p01532 medium B0827-2703 -- 8 256 -- 05:59 Q --
-288638.sdb p01532 medium B0827-2703 -- 8 256 -- 05:59 Q --</tt></pre>
-</div></div>
-<div class="paragraph"><p>The following section is a summary of the important <tt>sites.xml</tt>
-attributes for running apps on Cray systems. Many of these attributes
-can be set the same for all Swift users of a given system; only a few
-of the attributes need be overridden by users. We explain these
-attributes in detail here to show the degree of control afforded by
-Swift over application execution. Most users will use templates for a
-given Cray system, only changing a few parameters to meet any unique
-needs of their application workflows.</p></div>
-<div class="paragraph"><p>The additional attributes in the <tt>sites.xml</tt> file (described here
-without their XML formatting) specify that Swift should run
-applications on Raven in the following manner:</p></div>
-<div class="paragraph"><p><tt>execution provider coaster, jobmanager local:pbs</tt> specifies that
-Swift should run apps using its "coaster" provider, which submits
-"pilot jobs" using qsub. These pilot jobs hold on to compute nodes and
-allow Swift to run many app invocations within a single job. This
-mechanism is described in
-<a href="http://www.swift-lang.org/papers/UCC-coasters.pdf">this paper from UCC-2011</a>.</p></div>
-<div class="paragraph"><p><tt>profile</tt> tags specify additional attributes for the execution
-provider. (A "provider" is like a driver which knows how to handle
-site-specific aspects of app execution). The attributes are grouped
-into various "namespaces", but we can ignore this for now).</p></div>
-<div class="paragraph"><p>The <tt>env</tt> key <tt>PATHPREFIX</tt> specifies that our tutorial <tt>app</tt> directory
-(<tt>../app</tt>) will be placed at the front of PATH to locate the app on
-the compute node.</p></div>
-<div class="paragraph"><p><tt>queue small</tt> specifies that pilot (coaster) jobs to run apps will be
-submitted to Raven’s <tt>small</tt> queue.</p></div>
-<div class="paragraph"><p><tt>providerAttributes pbs.aprun;pbs.mpp;depth=32</tt> specifies some
-Cray-specific attributes: that jobs should use Cray-specific PBS "mpp"
-resource attributes (eg <tt>mppwidth</tt> and <tt>mppnppn</tt>) and an mppdepth of
-32 (because we will be running one coaster process per node, and
-Raven’s XE6 dual IL-16 nodes have a depth of 32 processing elements
-(PEs).</p></div>
-<div class="paragraph"><p><tt>jobsPerNode 32</tt> tells Swift that each coaster should run up to 32
-concurrent apps. This can be reduced to place fewer apps per node, eg
-if each app needs more memory (or, rarely, greater than 32, e.g. if the apps are
-IO-bound or for benchmark experiments, etc).</p></div>
-<div class="paragraph"><p><tt>slots 2</tt> specifies that Swift will run up to 2 concurrent PBS jobs,
-and <tt>maxNodes 1</tt> specifies that each of these jobs will request only 1
-compute node.</p></div>
-<div class="paragraph"><p><tt>maxWallTime 00:01:00</tt> specifies that Swift should allow each app to
-run for up to one minute of walltime within the larger pilot job. In
-this example Swift will dynamically determine the total PBS walltime
-needed for the pilot job, but this can be specified manually using
-attributes <tt>maxtime</tt> along with <tt>highOverAllocation</tt> and
-<tt>lowOverAllocation</tt>.</p></div>
-<div class="paragraph"><p><tt>jobThrottle 3.20</tt> specifies that Swift should allow up to 320 apps to
-run on the <tt>raven</tt> site at once. This is typically set to a number
-greater than or equal to the number of slots x compute nodes x apps
-per node (<tt>jobsPerNode</tt> attribute).</p></div>
-<div class="paragraph"><p><tt>initialscore 10000</tt> is specified to override Swift’s automatic
-throttling, and forces an actual throttle value of approximately
-(specifically 1 over) <tt>jobThrottle</tt> * 100 to be used.</p></div>
-<div class="paragraph"><p>The last two attributes specify where and how Swift should perform
-data management. <tt>workdirectory /lus/scratch/{env.USER}/swiftwork</tt>
-specifies where the Swift "application execution sanbox directory"
-used for each app will be located. In some situations this can be a
-directory local to the compute node (eg, for Cray systems, <tt>/dev/shm</tt>
-or <tt>/tmp</tt>, if those are writable by user jobs and the nodes have
-sufficient space in these RAM-based filesystems).</p></div>
-<div class="paragraph"><p>Finally, <tt>stagingMethod sfs</tt> specifies that Swift will copy data to
-and from the shared file system to the application sandbox
-directories.</p></div>
-</div>
-</div>
-</div>
-<div id="footnotes"><hr /></div>
-<div id="footer">
-<div id="footer-text">
-Last updated 2013-09-11 13:13:14 CDT
-</div>
-</div>
-</body>
-</html>
Deleted: SwiftApps/tryswift/scripts/p4.swift
===================================================================
--- SwiftApps/tryswift/scripts/p4.swift 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/p4.swift 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,29 +0,0 @@
-type file;
-
-app (file out, file log) simulation (int sim_steps, int sim_range, int sim_values)
-{
- simulate "--timesteps" sim_steps "--range" sim_range "--nvalues" sim_values stdout=@out stderr=@log;
-}
-
-app (file out, file log) analyze (file s[])
-{
- stats filenames(s) stdout=@out stderr=@log;
-}
-
-int nsim = toInt(arg("nsim", "10"));
-int steps = toInt(arg("steps", "1"));
-int range = toInt(arg("range", "100"));
-int values = toInt(arg("values", "5"));
-
-file sims[];
-
-foreach i in [0:nsim-1] {
- file simout <single_file_mapper; file=strcat("output/sim_",i,".out")>;
- file simlog <single_file_mapper; file=strcat("output/sim_",i,".log")>;
- (simout,simlog) = simulation(steps,range,values);
- sims[i] = simout;
-}
-
-file stats_out<"output/average.out">;
-file stats_log<"output/average.log">;
-(stats_out, stats_log) = analyze(sims);
Deleted: SwiftApps/tryswift/scripts/p4.txt
===================================================================
--- SwiftApps/tryswift/scripts/p4.txt 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/p4.txt 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1 +0,0 @@
-p4
Deleted: SwiftApps/tryswift/scripts/p5.html
===================================================================
--- SwiftApps/tryswift/scripts/p5.html 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/p5.html 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,716 +0,0 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
- "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
-<head>
-<meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
-<meta name="generator" content="AsciiDoc 8.6.4" />
-<title>Part 5: Controlling the compute-node pools where applications run</title>
-<style type="text/css">
-/* Sans-serif font. */
-h1, h2, h3, h4, h5, h6,
-div.title, caption.title,
-thead, p.table.header,
-div#toctitle,
-span#author, span#revnumber, span#revdate, span#revremark,
-div#footer {
- font-family: Arial,Helvetica,sans-serif;
-}
-
-/* Serif font. */
-div.sectionbody {
- font-family: Georgia,"Times New Roman",Times,serif;
-}
-
-/* Monospace font. */
-tt {
- font-size: inherit;
-}
-
-body {
- margin: 1em 5% 1em 5%;
-}
-
-a {
- color: blue;
- text-decoration: underline;
-}
-a:visited {
- color: fuchsia;
-}
-
-em {
- font-style: italic;
- color: navy;
-}
-
-strong {
- font-weight: bold;
- color: #083194;
-}
-
-tt {
- font-size: inherit;
- color: navy;
-}
-
-h1, h2, h3, h4, h5, h6 {
- color: #527bbd;
- margin-top: 1.2em;
- margin-bottom: 0.5em;
- line-height: 1.3;
-}
-
-h1, h2, h3 {
- border-bottom: 2px solid silver;
-}
-h2 {
- padding-top: 0.5em;
-}
-h3 {
- float: left;
-}
-h3 + * {
- clear: left;
-}
-
-div.sectionbody {
- margin-left: 0;
-}
-
-hr {
- border: 1px solid silver;
-}
-
-p {
- margin-top: 0.5em;
- margin-bottom: 0.5em;
-}
-
-ul, ol, li > p {
- margin-top: 0;
-}
-ul > li { color: #aaa; }
-ul > li > * { color: black; }
-
-pre {
- padding: 0;
- margin: 0;
-}
-
-span#author {
- color: #527bbd;
- font-weight: bold;
- font-size: 1.1em;
-}
-span#email {
-}
-span#revnumber, span#revdate, span#revremark {
-}
-
-div#footer {
- font-size: small;
- border-top: 2px solid silver;
- padding-top: 0.5em;
- margin-top: 4.0em;
-}
-div#footer-text {
- float: left;
- padding-bottom: 0.5em;
-}
-div#footer-badges {
- float: right;
- padding-bottom: 0.5em;
-}
-
-div#preamble {
- margin-top: 1.5em;
- margin-bottom: 1.5em;
-}
-div.tableblock, div.imageblock, div.exampleblock, div.verseblock,
-div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
-div.admonitionblock {
- margin-top: 1.0em;
- margin-bottom: 1.5em;
-}
-div.admonitionblock {
- margin-top: 2.0em;
- margin-bottom: 2.0em;
- margin-right: 10%;
- color: #606060;
-}
-
-div.content { /* Block element content. */
- padding: 0;
-}
-
-/* Block element titles. */
-div.title, caption.title {
- color: #527bbd;
- font-weight: bold;
- text-align: left;
- margin-top: 1.0em;
- margin-bottom: 0.5em;
-}
-div.title + * {
- margin-top: 0;
-}
-
-td div.title:first-child {
- margin-top: 0.0em;
-}
-div.content div.title:first-child {
- margin-top: 0.0em;
-}
-div.content + div.title {
- margin-top: 0.0em;
-}
-
-div.sidebarblock > div.content {
- background: #ffffee;
- border: 1px solid #dddddd;
- border-left: 4px solid #f0f0f0;
- padding: 0.5em;
-}
-
-div.listingblock > div.content {
- border: 1px solid #dddddd;
- border-left: 5px solid #f0f0f0;
- background: #f8f8f8;
- padding: 0.5em;
-}
-
-div.quoteblock, div.verseblock {
- padding-left: 1.0em;
- margin-left: 1.0em;
- margin-right: 10%;
- border-left: 5px solid #f0f0f0;
- color: #777777;
-}
-
-div.quoteblock > div.attribution {
- padding-top: 0.5em;
- text-align: right;
-}
-
-div.verseblock > pre.content {
- font-family: inherit;
- font-size: inherit;
-}
-div.verseblock > div.attribution {
- padding-top: 0.75em;
- text-align: left;
-}
-/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
-div.verseblock + div.attribution {
- text-align: left;
-}
-
-div.admonitionblock .icon {
- vertical-align: top;
- font-size: 1.1em;
- font-weight: bold;
- text-decoration: underline;
- color: #527bbd;
- padding-right: 0.5em;
-}
-div.admonitionblock td.content {
- padding-left: 0.5em;
- border-left: 3px solid #dddddd;
-}
-
-div.exampleblock > div.content {
- border-left: 3px solid #dddddd;
- padding-left: 0.5em;
-}
-
-div.imageblock div.content { padding-left: 0; }
-span.image img { border-style: none; }
-a.image:visited { color: white; }
-
-dl {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-dt {
- margin-top: 0.5em;
- margin-bottom: 0;
- font-style: normal;
- color: navy;
-}
-dd > *:first-child {
- margin-top: 0.1em;
-}
-
-ul, ol {
- list-style-position: outside;
-}
-ol.arabic {
- list-style-type: decimal;
-}
-ol.loweralpha {
- list-style-type: lower-alpha;
-}
-ol.upperalpha {
- list-style-type: upper-alpha;
-}
-ol.lowerroman {
- list-style-type: lower-roman;
-}
-ol.upperroman {
- list-style-type: upper-roman;
-}
-
-div.compact ul, div.compact ol,
-div.compact p, div.compact p,
-div.compact div, div.compact div {
- margin-top: 0.1em;
- margin-bottom: 0.1em;
-}
-
-div.tableblock > table {
- border: 3px solid #527bbd;
-}
-thead, p.table.header {
- font-weight: bold;
- color: #527bbd;
-}
-tfoot {
- font-weight: bold;
-}
-td > div.verse {
- white-space: pre;
-}
-p.table {
- margin-top: 0;
-}
-/* Because the table frame attribute is overriden by CSS in most browsers. */
-div.tableblock > table[frame="void"] {
- border-style: none;
-}
-div.tableblock > table[frame="hsides"] {
- border-left-style: none;
- border-right-style: none;
-}
-div.tableblock > table[frame="vsides"] {
- border-top-style: none;
- border-bottom-style: none;
-}
-
-
-div.hdlist {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-div.hdlist tr {
- padding-bottom: 15px;
-}
-dt.hdlist1.strong, td.hdlist1.strong {
- font-weight: bold;
-}
-td.hdlist1 {
- vertical-align: top;
- font-style: normal;
- padding-right: 0.8em;
- color: navy;
-}
-td.hdlist2 {
- vertical-align: top;
-}
-div.hdlist.compact tr {
- margin: 0;
- padding-bottom: 0;
-}
-
-.comment {
- background: yellow;
-}
-
-.footnote, .footnoteref {
- font-size: 0.8em;
-}
-
-span.footnote, span.footnoteref {
- vertical-align: super;
-}
-
-#footnotes {
- margin: 20px 0 20px 0;
- padding: 7px 0 0 0;
-}
-
-#footnotes div.footnote {
- margin: 0 0 5px 0;
-}
-
-#footnotes hr {
- border: none;
- border-top: 1px solid silver;
- height: 1px;
- text-align: left;
- margin-left: 0;
- width: 20%;
- min-width: 100px;
-}
-
-div.colist td {
- padding-right: 0.5em;
- padding-bottom: 0.3em;
- vertical-align: top;
-}
-div.colist td img {
- margin-top: 0.3em;
-}
-
- at media print {
- div#footer-badges { display: none; }
-}
-
-div#toc {
- margin-bottom: 2.5em;
-}
-
-div#toctitle {
- color: #527bbd;
- font-size: 1.1em;
- font-weight: bold;
- margin-top: 1.0em;
- margin-bottom: 0.1em;
-}
-
-div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
- margin-top: 0;
- margin-bottom: 0;
-}
-div.toclevel2 {
- margin-left: 2em;
- font-size: 0.9em;
-}
-div.toclevel3 {
- margin-left: 4em;
- font-size: 0.9em;
-}
-div.toclevel4 {
- margin-left: 6em;
- font-size: 0.9em;
-}
-
-span.aqua { color: aqua; }
-span.black { color: black; }
-span.blue { color: blue; }
-span.fuchsia { color: fuchsia; }
-span.gray { color: gray; }
-span.green { color: green; }
-span.lime { color: lime; }
-span.maroon { color: maroon; }
-span.navy { color: navy; }
-span.olive { color: olive; }
-span.purple { color: purple; }
-span.red { color: red; }
-span.silver { color: silver; }
-span.teal { color: teal; }
-span.white { color: white; }
-span.yellow { color: yellow; }
-
-span.aqua-background { background: aqua; }
-span.black-background { background: black; }
-span.blue-background { background: blue; }
-span.fuchsia-background { background: fuchsia; }
-span.gray-background { background: gray; }
-span.green-background { background: green; }
-span.lime-background { background: lime; }
-span.maroon-background { background: maroon; }
-span.navy-background { background: navy; }
-span.olive-background { background: olive; }
-span.purple-background { background: purple; }
-span.red-background { background: red; }
-span.silver-background { background: silver; }
-span.teal-background { background: teal; }
-span.white-background { background: white; }
-span.yellow-background { background: yellow; }
-
-span.big { font-size: 2em; }
-span.small { font-size: 0.6em; }
-a:link { color:navy; }
-a:visited { color:navy; }
-
-.monospaced, code, pre {
- font-family: "Courier New", Courier, monospace;
- font-size: medium; /* inherit; */
- color: black;
- padding: 0;
- margin: 0;
-}
-
-/*
- background: #f8f8f8;
- border: 1px solid #dddddd;
- border-left: 5px solid #f0f0f0;
- margin-right: 10%;
-*/
-
-div.listingblock > div.content {
- padding: 0.5em;
- background: none;
- border: none;
- border-left: none;
- margin-right: none;
-}
-
-</style>
-<script type="text/javascript">
-/*<+'])');
- // Function that scans the DOM tree for header elements (the DOM2
- // nodeIterator API would be a better technique but not supported by all
- // browsers).
- var iterate = function (el) {
- for (var i = el.firstChild; i != null; i = i.nextSibling) {
- if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
- var mo = re.exec(i.tagName);
- if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
- result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
- }
- iterate(i);
- }
- }
- }
- iterate(el);
- return result;
- }
-
- var toc = document.getElementById("toc");
- var entries = tocEntries(document.getElementById("content"), toclevels);
- for (var i = 0; i < entries.length; ++i) {
- var entry = entries[i];
- if (entry.element.id == "")
- entry.element.id = "_toc_" + i;
- var a = document.createElement("a");
- a.href = "#" + entry.element.id;
- a.appendChild(document.createTextNode(entry.text));
- var div = document.createElement("div");
- div.appendChild(a);
- div.className = "toclevel" + entry.toclevel;
- toc.appendChild(div);
- }
- if (entries.length == 0)
- toc.parentNode.removeChild(toc);
-},
-
-
-/////////////////////////////////////////////////////////////////////
-// Footnotes generator
-/////////////////////////////////////////////////////////////////////
-
-/* Based on footnote generation code from:
- * http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
- */
-
-footnotes: function () {
- var cont = document.getElementById("content");
- var noteholder = document.getElementById("footnotes");
- var spans = cont.getElementsByTagName("span");
- var refs = {};
- var n = 0;
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnote") {
- n++;
- // Use [\s\S] in place of . so multi-line matches work.
- // Because JavaScript has no s (dotall) regex flag.
- note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
- noteholder.innerHTML +=
- "<div class='footnote' id='_footnote_" + n + "'>" +
- "<a href='#_footnoteref_" + n + "' title='Return to text'>" +
- n + "</a>. " + note + "</div>";
- spans[i].innerHTML =
- "[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- var id =spans[i].getAttribute("id");
- if (id != null) refs["#"+id] = n;
- }
- }
- if (n == 0)
- noteholder.parentNode.removeChild(noteholder);
- else {
- // Process footnoterefs.
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnoteref") {
- var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
- href = href.match(/#.*/)[0]; // Because IE return full URL.
- n = refs[href];
- spans[i].innerHTML =
- "[<a href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- }
- }
- }
-}
-
-}
-/*]]>*/
-</script>
-</head>
-<body class="article" style="max-width:800px">
-<div id="header">
-<h1>Part 5: Controlling the compute-node pools where applications run</h1>
-</div>
-<div id="content">
-<div id="preamble">
-<div class="sectionbody">
-<div class="paragraph"><p>In this section we’ll use the script <tt>p5.swift</tt>, very similar to
-<tt>p4.swift</tt> of the prior section, to show how we can route apps to
-specific sites, and also how we can make multiple pools of resources
-(on the same or on different computer systems) available to run a
-single Swift script.</p></div>
-<div class="paragraph"><p>First, lets specify that the analysis app <tt>stats.sh</tt> should be run on
-the local login node instead of on the cluster. This is done simply by
-change the site name field of the analyze app in the apps file:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ cat apps
-raven simulate simulate.sh
-localhost stats stats.sh</tt></pre>
-</div></div>
-<div class="paragraph"><p>Running this with <tt>swift p5.swift</tt> we see:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ grep "on node:" output/*.log
-output/average.log:Running on node: raven
-output/sim_0.log:Running on node: nid00029
-output/sim_1.log:Running on node: nid00029
-output/sim_2.log:Running on node: nid00029
-output/sim_3.log:Running on node: nid00029
-output/sim_4.log:Running on node: nid00029
-output/sim_5.log:Running on node: nid00029
-output/sim_6.log:Running on node: nid00029
-output/sim_7.log:Running on node: nid00029
-output/sim_8.log:Running on node: nid00029
-output/sim_9.log:Running on node: nid00029</tt></pre>
-</div></div>
-<div class="paragraph"><p>Now lets make further use of Swift’s ability to route specific apps to
-specific pools of resources. The Cray Raven system has two node types,
-XE6 32-core 2 x IL-16, and XK7 16-core 1 x IL-16 plus one GPU. Each
-"pool" of nodes has different queue characteristics. We can define
-these differences to Swift as two separate pools, and then spread the
-load of executing a large ensemble of simulations across all the
-pools. (And we’ll continue to run the analysis script on a third pool,
-comprising the single login host.)</p></div>
-<div class="paragraph"><p>We use the following <tt>apps</tt> file:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ cat multipools
-raven simulate simulate.sh
-ravenGPU simulate simulate.sh
-localhost stats stats.sh</tt></pre>
-</div></div>
-<div class="paragraph"><p>and we adjust the sites file to specify 4-node jobs in the Raven pool:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt> <profile namespace="globus" key="maxNodes">4</profile>
- <profile namespace="globus" key="nodeGranularity">4</profile></tt></pre>
-</div></div>
-<div class="paragraph"><p>This results in these PBS jobs:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>p01532 at raven:~> qstat -u $USER
-
-Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
---------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
-288687.sdb p01532 small B0827-3406 9919 4 128 -- 00:59 R 00:00
-288688.sdb p01532 gpu_node B0827-3406 9931 6 96 -- 00:59 R 00:00</tt></pre>
-</div></div>
-<div class="olist lowerroman"><ol class="lowerroman">
-<li>
-<p>
-and achieves the following parallelism (of about 224 concurrent app tasks):
-</p>
-</li>
-</ol></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ swift -tc.file multipools p5.swift -nsim=1000 -steps=3
-Swift 0.94.1 RC2 swift-r6895 cog-r3765
-
-RunID: 20130827-1829-o3h6mht5
-Progress: time: Tue, 27 Aug 2013 18:29:31 -0500
-Progress: time: Tue, 27 Aug 2013 18:29:32 -0500 Initializing:997
-Progress: time: Tue, 27 Aug 2013 18:29:34 -0500 Selecting site:774 Submitting:225 Submitted:1
-Progress: time: Tue, 27 Aug 2013 18:29:35 -0500 Selecting site:774 Stage in:1 Submitted:225
-Progress: time: Tue, 27 Aug 2013 18:29:36 -0500 Selecting site:774 Stage in:1 Submitted:37 Active:188
-Progress: time: Tue, 27 Aug 2013 18:29:39 -0500 Selecting site:774 Submitted:2 Active:223 Stage out:1
-Progress: time: Tue, 27 Aug 2013 18:29:40 -0500 Selecting site:750 Submitted:17 Active:208 Stage out:1 Finished successfully:24
-Progress: time: Tue, 27 Aug 2013 18:29:41 -0500 Selecting site:640 Stage in:1 Submitted:51 Active:174 Finished successfully:134
-Progress: time: Tue, 27 Aug 2013 18:29:42 -0500 Selecting site:551 Submitted:11 Active:214 Stage out:1 Finished successfully:223
-Progress: time: Tue, 27 Aug 2013 18:29:43 -0500 Selecting site:542 Submitted:2 Active:223 Stage out:1 Finished successfully:232
-Progress: time: Tue, 27 Aug 2013 18:29:44 -0500 Selecting site:511 Submitting:1 Submitted:19 Active:206 Finished successfully:263
-Progress: time: Tue, 27 Aug 2013 18:29:45 -0500 Selecting site:463 Stage in:1 Submitted:43 Active:182 Finished successfully:311
-Progress: time: Tue, 27 Aug 2013 18:29:46 -0500 Selecting site:367 Submitting:1 Submitted:38 Active:186 Stage out:1 Finished successfully:407
-Progress: time: Tue, 27 Aug 2013 18:29:47 -0500 Selecting site:309 Submitted:2 Active:223 Stage out:1 Finished successfully:465
-Progress: time: Tue, 27 Aug 2013 18:29:48 -0500 Selecting site:300 Submitted:2 Active:223 Stage out:1 Finished successfully:474
-Progress: time: Tue, 27 Aug 2013 18:29:50 -0500 Selecting site:259 Submitted:11 Active:214 Stage out:1 Finished successfully:515
-Progress: time: Tue, 27 Aug 2013 18:29:51 -0500 Selecting site:201 Stage in:1 Submitted:39 Active:186 Finished successfully:573
-Progress: time: Tue, 27 Aug 2013 18:29:52 -0500 Selecting site:80 Submitted:42 Active:184 Finished successfully:694
-Progress: time: Tue, 27 Aug 2013 18:29:53 -0500 Selecting site:54 Submitted:2 Active:223 Stage out:1 Finished successfully:720
-Progress: time: Tue, 27 Aug 2013 18:29:54 -0500 Selecting site:32 Submitted:4 Active:220 Stage out:1 Finished successfully:743
-Progress: time: Tue, 27 Aug 2013 18:29:55 -0500 Submitted:3 Active:216 Stage out:1 Finished successfully:780
-Progress: time: Tue, 27 Aug 2013 18:29:56 -0500 Stage in:1 Active:143 Finished successfully:856
-Progress: time: Tue, 27 Aug 2013 18:29:57 -0500 Active:38 Stage out:1 Finished successfully:961
-Progress: time: Tue, 27 Aug 2013 18:29:58 -0500 Active:8 Stage out:1 Finished successfully:991
-Progress: time: Tue, 27 Aug 2013 18:29:59 -0500 Stage out:1 Finished successfully:999
-Progress: time: Tue, 27 Aug 2013 18:30:01 -0500 Stage in:1 Finished successfully:1000
-Progress: time: Tue, 27 Aug 2013 18:30:02 -0500 Active:1 Finished successfully:1000
-Progress: time: Tue, 27 Aug 2013 18:30:06 -0500 Stage out:1 Finished successfully:1000
-Final status: Tue, 27 Aug 2013 18:30:07 -0500 Finished successfully:1001</tt></pre>
-</div></div>
-</div>
-</div>
-</div>
-<div id="footnotes"><hr /></div>
-<div id="footer">
-<div id="footer-text">
-Last updated 2013-09-11 13:13:27 CDT
-</div>
-</div>
-</body>
-</html>
Deleted: SwiftApps/tryswift/scripts/p5.swift
===================================================================
--- SwiftApps/tryswift/scripts/p5.swift 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/p5.swift 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,29 +0,0 @@
-type file;
-
-app (file out, file log) simulation (int sim_steps, int sim_range, int sim_values)
-{
- simulate "--timesteps" sim_steps "--range" sim_range "--nvalues" sim_values stdout=@out stderr=@log;
-}
-
-app (file out, file log) analyze (file s[])
-{
- stats filenames(s) stdout=@out stderr=@log;
-}
-
-int nsim = toInt(arg("nsim", "10"));
-int steps = toInt(arg("steps", "1"));
-int range = toInt(arg("range", "100"));
-int values = toInt(arg("values", "5"));
-
-file sims[];
-
-foreach i in [0:nsim-1] {
- file simout <single_file_mapper; file=strcat("output/sim_",i,".out")>;
- file simlog <single_file_mapper; file=strcat("output/sim_",i,".log")>;
- (simout,simlog) = simulation(steps,range,values);
- sims[i] = simout;
-}
-
-file stats_out<"output/average.out">;
-file stats_log<"output/average.log">;
-(stats_out, stats_log) = analyze(sims);
Deleted: SwiftApps/tryswift/scripts/p5.txt
===================================================================
--- SwiftApps/tryswift/scripts/p5.txt 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/p5.txt 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1 +0,0 @@
-p5
Deleted: SwiftApps/tryswift/scripts/p6.html
===================================================================
--- SwiftApps/tryswift/scripts/p6.html 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/p6.html 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,797 +0,0 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
- "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
-<head>
-<meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
-<meta name="generator" content="AsciiDoc 8.6.4" />
-<title></title>
-<style type="text/css">
-/* Sans-serif font. */
-h1, h2, h3, h4, h5, h6,
-div.title, caption.title,
-thead, p.table.header,
-div#toctitle,
-span#author, span#revnumber, span#revdate, span#revremark,
-div#footer {
- font-family: Arial,Helvetica,sans-serif;
-}
-
-/* Serif font. */
-div.sectionbody {
- font-family: Georgia,"Times New Roman",Times,serif;
-}
-
-/* Monospace font. */
-tt {
- font-size: inherit;
-}
-
-body {
- margin: 1em 5% 1em 5%;
-}
-
-a {
- color: blue;
- text-decoration: underline;
-}
-a:visited {
- color: fuchsia;
-}
-
-em {
- font-style: italic;
- color: navy;
-}
-
-strong {
- font-weight: bold;
- color: #083194;
-}
-
-tt {
- font-size: inherit;
- color: navy;
-}
-
-h1, h2, h3, h4, h5, h6 {
- color: #527bbd;
- margin-top: 1.2em;
- margin-bottom: 0.5em;
- line-height: 1.3;
-}
-
-h1, h2, h3 {
- border-bottom: 2px solid silver;
-}
-h2 {
- padding-top: 0.5em;
-}
-h3 {
- float: left;
-}
-h3 + * {
- clear: left;
-}
-
-div.sectionbody {
- margin-left: 0;
-}
-
-hr {
- border: 1px solid silver;
-}
-
-p {
- margin-top: 0.5em;
- margin-bottom: 0.5em;
-}
-
-ul, ol, li > p {
- margin-top: 0;
-}
-ul > li { color: #aaa; }
-ul > li > * { color: black; }
-
-pre {
- padding: 0;
- margin: 0;
-}
-
-span#author {
- color: #527bbd;
- font-weight: bold;
- font-size: 1.1em;
-}
-span#email {
-}
-span#revnumber, span#revdate, span#revremark {
-}
-
-div#footer {
- font-size: small;
- border-top: 2px solid silver;
- padding-top: 0.5em;
- margin-top: 4.0em;
-}
-div#footer-text {
- float: left;
- padding-bottom: 0.5em;
-}
-div#footer-badges {
- float: right;
- padding-bottom: 0.5em;
-}
-
-div#preamble {
- margin-top: 1.5em;
- margin-bottom: 1.5em;
-}
-div.tableblock, div.imageblock, div.exampleblock, div.verseblock,
-div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
-div.admonitionblock {
- margin-top: 1.0em;
- margin-bottom: 1.5em;
-}
-div.admonitionblock {
- margin-top: 2.0em;
- margin-bottom: 2.0em;
- margin-right: 10%;
- color: #606060;
-}
-
-div.content { /* Block element content. */
- padding: 0;
-}
-
-/* Block element titles. */
-div.title, caption.title {
- color: #527bbd;
- font-weight: bold;
- text-align: left;
- margin-top: 1.0em;
- margin-bottom: 0.5em;
-}
-div.title + * {
- margin-top: 0;
-}
-
-td div.title:first-child {
- margin-top: 0.0em;
-}
-div.content div.title:first-child {
- margin-top: 0.0em;
-}
-div.content + div.title {
- margin-top: 0.0em;
-}
-
-div.sidebarblock > div.content {
- background: #ffffee;
- border: 1px solid #dddddd;
- border-left: 4px solid #f0f0f0;
- padding: 0.5em;
-}
-
-div.listingblock > div.content {
- border: 1px solid #dddddd;
- border-left: 5px solid #f0f0f0;
- background: #f8f8f8;
- padding: 0.5em;
-}
-
-div.quoteblock, div.verseblock {
- padding-left: 1.0em;
- margin-left: 1.0em;
- margin-right: 10%;
- border-left: 5px solid #f0f0f0;
- color: #777777;
-}
-
-div.quoteblock > div.attribution {
- padding-top: 0.5em;
- text-align: right;
-}
-
-div.verseblock > pre.content {
- font-family: inherit;
- font-size: inherit;
-}
-div.verseblock > div.attribution {
- padding-top: 0.75em;
- text-align: left;
-}
-/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
-div.verseblock + div.attribution {
- text-align: left;
-}
-
-div.admonitionblock .icon {
- vertical-align: top;
- font-size: 1.1em;
- font-weight: bold;
- text-decoration: underline;
- color: #527bbd;
- padding-right: 0.5em;
-}
-div.admonitionblock td.content {
- padding-left: 0.5em;
- border-left: 3px solid #dddddd;
-}
-
-div.exampleblock > div.content {
- border-left: 3px solid #dddddd;
- padding-left: 0.5em;
-}
-
-div.imageblock div.content { padding-left: 0; }
-span.image img { border-style: none; }
-a.image:visited { color: white; }
-
-dl {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-dt {
- margin-top: 0.5em;
- margin-bottom: 0;
- font-style: normal;
- color: navy;
-}
-dd > *:first-child {
- margin-top: 0.1em;
-}
-
-ul, ol {
- list-style-position: outside;
-}
-ol.arabic {
- list-style-type: decimal;
-}
-ol.loweralpha {
- list-style-type: lower-alpha;
-}
-ol.upperalpha {
- list-style-type: upper-alpha;
-}
-ol.lowerroman {
- list-style-type: lower-roman;
-}
-ol.upperroman {
- list-style-type: upper-roman;
-}
-
-div.compact ul, div.compact ol,
-div.compact p, div.compact p,
-div.compact div, div.compact div {
- margin-top: 0.1em;
- margin-bottom: 0.1em;
-}
-
-div.tableblock > table {
- border: 3px solid #527bbd;
-}
-thead, p.table.header {
- font-weight: bold;
- color: #527bbd;
-}
-tfoot {
- font-weight: bold;
-}
-td > div.verse {
- white-space: pre;
-}
-p.table {
- margin-top: 0;
-}
-/* Because the table frame attribute is overriden by CSS in most browsers. */
-div.tableblock > table[frame="void"] {
- border-style: none;
-}
-div.tableblock > table[frame="hsides"] {
- border-left-style: none;
- border-right-style: none;
-}
-div.tableblock > table[frame="vsides"] {
- border-top-style: none;
- border-bottom-style: none;
-}
-
-
-div.hdlist {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-div.hdlist tr {
- padding-bottom: 15px;
-}
-dt.hdlist1.strong, td.hdlist1.strong {
- font-weight: bold;
-}
-td.hdlist1 {
- vertical-align: top;
- font-style: normal;
- padding-right: 0.8em;
- color: navy;
-}
-td.hdlist2 {
- vertical-align: top;
-}
-div.hdlist.compact tr {
- margin: 0;
- padding-bottom: 0;
-}
-
-.comment {
- background: yellow;
-}
-
-.footnote, .footnoteref {
- font-size: 0.8em;
-}
-
-span.footnote, span.footnoteref {
- vertical-align: super;
-}
-
-#footnotes {
- margin: 20px 0 20px 0;
- padding: 7px 0 0 0;
-}
-
-#footnotes div.footnote {
- margin: 0 0 5px 0;
-}
-
-#footnotes hr {
- border: none;
- border-top: 1px solid silver;
- height: 1px;
- text-align: left;
- margin-left: 0;
- width: 20%;
- min-width: 100px;
-}
-
-div.colist td {
- padding-right: 0.5em;
- padding-bottom: 0.3em;
- vertical-align: top;
-}
-div.colist td img {
- margin-top: 0.3em;
-}
-
- at media print {
- div#footer-badges { display: none; }
-}
-
-div#toc {
- margin-bottom: 2.5em;
-}
-
-div#toctitle {
- color: #527bbd;
- font-size: 1.1em;
- font-weight: bold;
- margin-top: 1.0em;
- margin-bottom: 0.1em;
-}
-
-div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
- margin-top: 0;
- margin-bottom: 0;
-}
-div.toclevel2 {
- margin-left: 2em;
- font-size: 0.9em;
-}
-div.toclevel3 {
- margin-left: 4em;
- font-size: 0.9em;
-}
-div.toclevel4 {
- margin-left: 6em;
- font-size: 0.9em;
-}
-
-span.aqua { color: aqua; }
-span.black { color: black; }
-span.blue { color: blue; }
-span.fuchsia { color: fuchsia; }
-span.gray { color: gray; }
-span.green { color: green; }
-span.lime { color: lime; }
-span.maroon { color: maroon; }
-span.navy { color: navy; }
-span.olive { color: olive; }
-span.purple { color: purple; }
-span.red { color: red; }
-span.silver { color: silver; }
-span.teal { color: teal; }
-span.white { color: white; }
-span.yellow { color: yellow; }
-
-span.aqua-background { background: aqua; }
-span.black-background { background: black; }
-span.blue-background { background: blue; }
-span.fuchsia-background { background: fuchsia; }
-span.gray-background { background: gray; }
-span.green-background { background: green; }
-span.lime-background { background: lime; }
-span.maroon-background { background: maroon; }
-span.navy-background { background: navy; }
-span.olive-background { background: olive; }
-span.purple-background { background: purple; }
-span.red-background { background: red; }
-span.silver-background { background: silver; }
-span.teal-background { background: teal; }
-span.white-background { background: white; }
-span.yellow-background { background: yellow; }
-
-span.big { font-size: 2em; }
-span.small { font-size: 0.6em; }
-a:link { color:navy; }
-a:visited { color:navy; }
-
-.monospaced, code, pre {
- font-family: "Courier New", Courier, monospace;
- font-size: medium; /* inherit; */
- color: black;
- padding: 0;
- margin: 0;
-}
-
-/*
- background: #f8f8f8;
- border: 1px solid #dddddd;
- border-left: 5px solid #f0f0f0;
- margin-right: 10%;
-*/
-
-div.listingblock > div.content {
- padding: 0.5em;
- background: none;
- border: none;
- border-left: none;
- margin-right: none;
-}
-
-</style>
-<script type="text/javascript">
-/*<+'])');
- // Function that scans the DOM tree for header elements (the DOM2
- // nodeIterator API would be a better technique but not supported by all
- // browsers).
- var iterate = function (el) {
- for (var i = el.firstChild; i != null; i = i.nextSibling) {
- if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
- var mo = re.exec(i.tagName);
- if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
- result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
- }
- iterate(i);
- }
- }
- }
- iterate(el);
- return result;
- }
-
- var toc = document.getElementById("toc");
- var entries = tocEntries(document.getElementById("content"), toclevels);
- for (var i = 0; i < entries.length; ++i) {
- var entry = entries[i];
- if (entry.element.id == "")
- entry.element.id = "_toc_" + i;
- var a = document.createElement("a");
- a.href = "#" + entry.element.id;
- a.appendChild(document.createTextNode(entry.text));
- var div = document.createElement("div");
- div.appendChild(a);
- div.className = "toclevel" + entry.toclevel;
- toc.appendChild(div);
- }
- if (entries.length == 0)
- toc.parentNode.removeChild(toc);
-},
-
-
-/////////////////////////////////////////////////////////////////////
-// Footnotes generator
-/////////////////////////////////////////////////////////////////////
-
-/* Based on footnote generation code from:
- * http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
- */
-
-footnotes: function () {
- var cont = document.getElementById("content");
- var noteholder = document.getElementById("footnotes");
- var spans = cont.getElementsByTagName("span");
- var refs = {};
- var n = 0;
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnote") {
- n++;
- // Use [\s\S] in place of . so multi-line matches work.
- // Because JavaScript has no s (dotall) regex flag.
- note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
- noteholder.innerHTML +=
- "<div class='footnote' id='_footnote_" + n + "'>" +
- "<a href='#_footnoteref_" + n + "' title='Return to text'>" +
- n + "</a>. " + note + "</div>";
- spans[i].innerHTML =
- "[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- var id =spans[i].getAttribute("id");
- if (id != null) refs["#"+id] = n;
- }
- }
- if (n == 0)
- noteholder.parentNode.removeChild(noteholder);
- else {
- // Process footnoterefs.
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnoteref") {
- var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
- href = href.match(/#.*/)[0]; // Because IE return full URL.
- n = refs[href];
- spans[i].innerHTML =
- "[<a href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- }
- }
- }
-}
-
-}
-/*]]>*/
-</script>
-</head>
-<body class="article" style="max-width:800px">
-<div id="header">
-</div>
-<div id="content">
-<div class="sect2">
-<h3 id="_part_6_specifying_more_complex_workflow_patterns">Part 6: Specifying more complex workflow patterns</h3>
-<div class="paragraph"><p>p6.swift expands the workflow pattern of p4.swift to add additional
-stages to the workflow. Here, we generate a dynamic seed value that
-will be used by all of the simulations, and for each simulation, we
-run an pre-processing application to generate a unique "bias
-file". This pattern is shown below, followed by the Swift script.</p></div>
-<div class="imageblock" style="text-align:center;">
-<div class="content">
-<img src="part06.png" alt="part06.png" />
-</div>
-</div>
-<div class="listingblock">
-<div class="title">p6.swift</div>
-<div class="content">
-<pre><tt>type file;
-
-# app() functions for application programs to be called:
-
-app (file out) genseed (int nseeds)
-{
- genseed "-r" 2000000 "-n" nseeds stdout=@out;
-}
-
-app (file out) genbias (int bias_range, int nvalues)
-{
- genbias "-r" bias_range "-n" nvalues stdout=@out;
-}
-
-app (file out, file log) simulation (int timesteps, int sim_range,
- file bias_file, int scale, int sim_count)
-{
- simulate "-t" timesteps "-r" sim_range "-B" @bias_file "-x" scale
- "-n" sim_count stdout=@out stderr=@log;
-}
-
-app (file out, file log) analyze (file s[])
-{
- stats @filenames(s) stdout=@out stderr=@log;
-}
-
-# Command line arguments
-
-int nsim = @toInt(@arg("nsim", "10")); # number of simulation programs to run
-int steps = @toInt(@arg("steps", "1")); # number of timesteps (seconds) per simulation
-int range = @toInt(@arg("range", "100")); # range of the generated random numbers
-int values = @toInt(@arg("values", "10")); # number of values generated per simulation
-
-# Main script and data
-
-tracef("\n*** Script parameters: nsim=%i range=%i num values=%i\n\n", nsim, range, values);
-
-file seedfile<"output/seed.dat">; # Dynamically generated bias for simulation ensemble
-seedfile = genseed(1);
-
-int seedval = readData(seedfile);
-tracef("Generated seed=%i\n", seedval);
-
-file sims[]; # Array of files to hold each simulation output
-
-foreach i in [0:nsim-1] {
- file biasfile <single_file_mapper; file=@strcat("output/bias_",i,".dat")>;
- file simout <single_file_mapper; file=@strcat("output/sim_",i,".out")>;
- file simlog <single_file_mapper; file=@strcat("output/sim_",i,".log")>;
- biasfile = genbias(1000, 20);
- (simout,simlog) = simulation(steps, range, biasfile, 1000000, values);
- sims[i] = simout;
-}
-
-file stats_out<"output/average.out">;
-file stats_log<"output/average.log">;
-(stats_out,stats_log) = analyze(sims);</tt></pre>
-</div></div>
-<div class="paragraph"><p>Note that the workflow is based on data flow dependencies: each simulation depends on the seed value, calculated in these two dependent statements:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>seedfile = genseed(1);
-int seedval = readData(seedfile);</tt></pre>
-</div></div>
-<div class="paragraph"><p>and on the bias file, computed and then consumed in these two dependent statements:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt> biasfile = genbias(1000, 20);
- (simout,simlog) = simulation(steps, range, biasfile, 1000000, values);</tt></pre>
-</div></div>
-<div class="paragraph"><p>To run:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ cd ../part06
-$ swift p6.swift</tt></pre>
-</div></div>
-<div class="paragraph"><p>The default parameters result in the following execution log:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ swift p6.swift
-Swift 0.94.1 RC2 swift-r6895 cog-r3765
-
-RunID: 20130827-1917-jvs4gqm5
-Progress: time: Tue, 27 Aug 2013 19:17:56 -0500
-
-*** Script parameters: nsim=10 range=100 num values=10
-
-Progress: time: Tue, 27 Aug 2013 19:17:57 -0500 Stage in:1 Submitted:10
-Generated seed=382537
-Progress: time: Tue, 27 Aug 2013 19:17:59 -0500 Active:9 Stage out:1 Finished successfully:11
-Final status: Tue, 27 Aug 2013 19:18:00 -0500 Finished successfully:22</tt></pre>
-</div></div>
-<div class="paragraph"><p>which produces the following output:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ ls -lrt output
-total 264
--rw-r--r-- 1 p01532 61532 9 Aug 27 19:17 seed.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_9.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_8.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_7.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_6.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_5.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_4.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_3.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_2.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_1.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_0.dat
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_9.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_9.log
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_8.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_7.out
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_6.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_6.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_5.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_5.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_4.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_4.log
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_1.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_8.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:18 sim_7.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_3.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:18 sim_3.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_2.out
--rw-r--r-- 1 p01532 61532 14898 Aug 27 19:18 sim_2.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_1.out
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_0.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:18 sim_0.log
--rw-r--r-- 1 p01532 61532 9 Aug 27 19:18 average.out
--rw-r--r-- 1 p01532 61532 14675 Aug 27 19:18 average.log</tt></pre>
-</div></div>
-<div class="paragraph"><p>Each sim_N.out file is the sum of its bias file plus newly "simulated" random output scaled by 1,000,000:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ cat output/bias_0.dat
- 302
- 489
- 81
- 582
- 664
- 290
- 839
- 258
- 506
- 310
- 293
- 508
- 88
- 261
- 453
- 187
- 26
- 198
- 402
- 555
-
-$ cat output/sim_0.out
-64000302
-38000489
-32000081
-12000582
-46000664
-36000290
-35000839
-22000258
-49000506
-75000310</tt></pre>
-</div></div>
-<div class="paragraph"><p>We produce 20 values in each bias file. Simulations of less than that
-number of values ignore the unneeded number, while simualtions of more
-than 20 will use the last bias number for all remoaining values past
-20. As an exercise, adjust the code to produce the same number of
-bias values as is needed for each simulation. As a further exercise,
-modify the script to generate a unique seed value for each simulation,
-which is a common practice in ensemble computations.</p></div>
-</div>
-</div>
-<div id="footnotes"><hr /></div>
-<div id="footer">
-<div id="footer-text">
-Last updated 2013-09-11 13:10:53 CDT
-</div>
-</div>
-</body>
-</html>
Deleted: SwiftApps/tryswift/scripts/p6.swift
===================================================================
--- SwiftApps/tryswift/scripts/p6.swift 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/p6.swift 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,57 +0,0 @@
-type file;
-
-# app() functions for application programs to be called:
-
-app (file out) genseed (int nseeds)
-{
- genseed "-r" 2000000 "-n" nseeds stdout=@out;
-}
-
-app (file out) genbias (int bias_range, int nvalues)
-{
- genbias "-r" bias_range "-n" nvalues stdout=@out;
-}
-
-app (file out, file log) simulation (int timesteps, int sim_range,
- file bias_file, int scale, int sim_count)
-{
- simulate "-t" timesteps "-r" sim_range "-B" @bias_file "-x" scale
- "-n" sim_count stdout=@out stderr=@log;
-}
-
-app (file out, file log) analyze (file s[])
-{
- stats filenames(s) stdout=@out stderr=@log;
-}
-
-# Command line arguments
-
-int nsim = toInt(arg("nsim", "10")); # number of simulation programs to run
-int steps = toInt(arg("steps", "1")); # number of timesteps (seconds) per simulation
-int range = toInt(arg("range", "100")); # range of the generated random numbers
-int values = toInt(arg("values", "10")); # number of values generated per simulation
-
-# Main script and data
-
-tracef("\n*** Script parameters: nsim=%i range=%i num values=%i\n\n", nsim, range, values);
-
-file seedfile<"output/seed.dat">; # Dynamically generated bias for simulation ensemble
-seedfile = genseed(1);
-
-int seedval = readData(seedfile);
-tracef("Generated seed=%i\n", seedval);
-
-file sims[]; # Array of files to hold each simulation output
-
-foreach i in [0:nsim-1] {
- file biasfile <single_file_mapper; file=strcat("output/bias_",i,".dat")>;
- file simout <single_file_mapper; file=strcat("output/sim_",i,".out")>;
- file simlog <single_file_mapper; file=strcat("output/sim_",i,".log")>;
- biasfile = genbias(1000, 20);
- (simout,simlog) = simulation(steps, range, biasfile, 1000000, values);
- sims[i] = simout;
-}
-
-file stats_out<"output/average.out">;
-file stats_log<"output/average.log">;
-(stats_out,stats_log) = analyze(sims);
Deleted: SwiftApps/tryswift/scripts/p6.txt
===================================================================
--- SwiftApps/tryswift/scripts/p6.txt 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/p6.txt 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1 +0,0 @@
-p6
Deleted: SwiftApps/tryswift/scripts/part04.png
===================================================================
(Binary files differ)
Deleted: SwiftApps/tryswift/scripts/part05.png
===================================================================
(Binary files differ)
Deleted: SwiftApps/tryswift/scripts/part06.png
===================================================================
(Binary files differ)
Deleted: SwiftApps/tryswift/scripts/part4.txt
===================================================================
--- SwiftApps/tryswift/scripts/part4.txt 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/part4.txt 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,249 +0,0 @@
-Part 4: Running a parallel ensemble on Cray compute nodes
-=========================================================
-
-`p4.swift` will run our mock "simulation"
-applications on Cray compute nodes. The script is similar to as
-`p3.swift`, but specifies that each simulation app invocation should
-additionally return the log file which the application writes to
-`stderr`.
-
-////
-
-FIXME: need to revise this figure: drop prog:
-
-, making the parallel portion of the script behave like this:
-
-image::part04.png[align="center"]
-
-.p4.swift
-----
-sys::[cat ../part04/p4.swift]
-----
-////
-
-Now when you run `swift p4.swift` you'll see that two types output
-files will placed in the `output/` directory: `sim_N.out` and
-`sim_N.log`. The log files provide data on the runtime environment of
-each app invocation. For example:
-
------
-$ cat output/sim_0.log
-Called as: /home/users/p01532/swift-cray-tutorial/app/simulate.sh: --timesteps 1 --range 100 --nvalues 5
-
-Start time: Tue Aug 27 12:17:43 CDT 2013
-Running on node: nid00018
-Running as user: uid=61532(p01532) gid=61532 groups=61532
-
-Simulation parameters:
-
-bias=0
-biasfile=none
-initseed=none
-log=yes
-paramfile=none
-range=100
-scale=1
-seedfile=none
-timesteps=1
-output width=8
-
-Environment:
-
-ALPS_APP_DEPTH=32
-ASSEMBLER_X86_64=/opt/cray/cce/8.2.0.173/cray-binutils/x86_64-unknown-linux-gnu/bin/as
-ASYNCPE_DIR=/opt/cray/xt-asyncpe/5.23.02
-ASYNCPE_VERSION=5.23.02
-...
------
-
-To tell Swift to run the apps on compute nodes, we specify in the
-`apps` file that the apps should be executed on the `raven` site
-(instead of the `localhost` site). We can specify the location of
-each app in the third field of the `apps` file, with either an
-absolute pathname or the name of an executable to be located in
-`PATH`). Here we use the latter form:
-
------
-$ cat apps
-raven simulate simulate.sh
-raven stats stats.sh
------
-
-You can experiment, for example, with an alternate version of stats.sh by specfying that app's location explicitly:
-
------
-$ cat apps
-raven simulate simulate.sh
-raven stats /home/users/p01532/bin/my-alt-stats.sh
------
-
-We can see that when we run many apps requesting a larger set of nodes (6), we are indeed running on the compute nodes:
------
-$ swift p4.swift -nsim=1000 -steps=1
-Swift 0.94.1 RC2 swift-r6895 cog-r3765
-
-RunID: 20130827-1638-t23ax37a
-Progress: time: Tue, 27 Aug 2013 16:38:11 -0500
-Progress: time: Tue, 27 Aug 2013 16:38:12 -0500 Initializing:966
-Progress: time: Tue, 27 Aug 2013 16:38:13 -0500 Selecting site:499 Submitting:500 Submitted:1
-Progress: time: Tue, 27 Aug 2013 16:38:14 -0500 Selecting site:499 Stage in:1 Submitted:500
-Progress: time: Tue, 27 Aug 2013 16:38:16 -0500 Selecting site:499 Submitted:405 Active:95 Stage out:1
-Progress: time: Tue, 27 Aug 2013 16:38:17 -0500 Selecting site:430 Submitted:434 Active:66 Stage out:1 Finished successfully:69
-Progress: time: Tue, 27 Aug 2013 16:38:18 -0500 Selecting site:388 Submitted:405 Active:95 Stage out:1 Finished successfully:111
-...
-Progress: time: Tue, 27 Aug 2013 16:38:30 -0500 Stage in:1 Submitted:93 Active:94 Finished successfully:812
-Progress: time: Tue, 27 Aug 2013 16:38:31 -0500 Submitted:55 Active:95 Stage out:1 Finished successfully:849
-Progress: time: Tue, 27 Aug 2013 16:38:32 -0500 Active:78 Stage out:1 Finished successfully:921
-Progress: time: Tue, 27 Aug 2013 16:38:34 -0500 Active:70 Stage out:1 Finished successfully:929
-Progress: time: Tue, 27 Aug 2013 16:38:37 -0500 Stage in:1 Finished successfully:1000
-Progress: time: Tue, 27 Aug 2013 16:38:38 -0500 Stage out:1 Finished successfully:1000
-Final status: Tue, 27 Aug 2013 16:38:38 -0500 Finished successfully:1001
-
-$ grep "on node:" output/*log | head
-output/sim_0.log:Running on node: nid00063
-output/sim_100.log:Running on node: nid00060
-output/sim_101.log:Running on node: nid00061
-output/sim_102.log:Running on node: nid00032
-output/sim_103.log:Running on node: nid00060
-output/sim_104.log:Running on node: nid00061
-output/sim_105.log:Running on node: nid00032
-output/sim_106.log:Running on node: nid00060
-output/sim_107.log:Running on node: nid00061
-output/sim_108.log:Running on node: nid00062
-
-$ grep "on node:" output/*log | awk '{print $4}' | sort | uniq -c
- 158 nid00032
- 156 nid00033
- 171 nid00060
- 178 nid00061
- 166 nid00062
- 171 nid00063
-$ hostname
-raven
-$ hostname -f
-nid00008
------
-
-Swift's `sites.xml` configuration file allows many parameters to
-specify how jobs should be run on a given cluster. Consider for
-example that Raven has several queues, each with limitiations on the
-size of jobs that can be run in them. All Raven queues will only run
-2 jobs per user at one. The Raven queue "small" will only allow up to
-4 nodes per job and 1 hours of walltime per job. The following
-site.xml parameters will allow us to match this:
-
------
- <profile namespace="globus" key="queue">small</profile>
- <profile namespace="globus" key="slots">2</profile>
- <profile namespace="globus" key="maxNodes">4</profile>
- <profile namespace="globus" key="nodeGranularity">4</profile>
------
-
-To run large jobs, we can specify:
-
------
- <profile namespace="globus" key="slots">2</profile>
- <profile namespace="globus" key="maxNodes">8</profile>
- <profile namespace="globus" key="nodeGranularity">8</profile>
- <profile namespace="karajan" key="jobThrottle">50.0</profile>
- <profile namespace="globus" key="maxTime">21600</profile>
- <profile namespace="globus" key="lowOverAllocation">10000</profile>
- <profile namespace="globus" key="highOverAllocation">10000</profile>
------
-
-This will enable 512 Swift apps (2 x 8 x 32) to run concurrently
-within 2 8-node jobs on Raven's 32-core nodes. It results in the
-following two PBS jobs submitted by Swift to "provision" compute nodes
-to run thousands of apps, 512 at a time:
-
------
-$ qstat -u $USER
-
-Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
---------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
-288637.sdb p01532 medium B0827-2703 -- 8 256 -- 05:59 Q --
-288638.sdb p01532 medium B0827-2703 -- 8 256 -- 05:59 Q --
------
-
-The following section is a summary of the important `sites.xml`
-attributes for running apps on Cray systems. Many of these attributes
-can be set the same for all Swift users of a given system; only a few
-of the attributes need be overridden by users. We explain these
-attributes in detail here to show the degree of control afforded by
-Swift over application execution. Most users will use templates for a
-given Cray system, only changing a few parameters to meet any unique
-needs of their application workflows.
-
-////
-.sites.xml
------
-sys::[egrep -v '<.xml|<config|</config' ../part04/sites.xml | cat -n ]
------
-////
-
-The additional attributes in the `sites.xml` file (described here
-without their XML formatting) specify that Swift should run
-applications on Raven in the following manner:
-
-`execution provider coaster, jobmanager local:pbs` specifies that
-Swift should run apps using its "coaster" provider, which submits
-"pilot jobs" using qsub. These pilot jobs hold on to compute nodes and
-allow Swift to run many app invocations within a single job. This
-mechanism is described in
-http://www.swift-lang.org/papers/UCC-coasters.pdf[this paper from UCC-2011].
-
-`profile` tags specify additional attributes for the execution
-provider. (A "provider" is like a driver which knows how to handle
-site-specific aspects of app execution). The attributes are grouped
-into various "namespaces", but we can ignore this for now).
-
-The `env` key `PATHPREFIX` specifies that our tutorial `app` directory
-(`../app`) will be placed at the front of PATH to locate the app on
-the compute node.
-
-`queue small` specifies that pilot (coaster) jobs to run apps will be
-submitted to Raven's `small` queue.
-
-`providerAttributes pbs.aprun;pbs.mpp;depth=32` specifies some
-Cray-specific attributes: that jobs should use Cray-specific PBS "mpp"
-resource attributes (eg `mppwidth` and `mppnppn`) and an mppdepth of
-32 (because we will be running one coaster process per node, and
-Raven's XE6 dual IL-16 nodes have a depth of 32 processing elements
-(PEs).
-
-`jobsPerNode 32` tells Swift that each coaster should run up to 32
-concurrent apps. This can be reduced to place fewer apps per node, eg
-if each app needs more memory (or, rarely, greater than 32, e.g. if the apps are
-IO-bound or for benchmark experiments, etc).
-
-`slots 2` specifies that Swift will run up to 2 concurrent PBS jobs,
-and `maxNodes 1` specifies that each of these jobs will request only 1
-compute node.
-
-`maxWallTime 00:01:00` specifies that Swift should allow each app to
-run for up to one minute of walltime within the larger pilot job. In
-this example Swift will dynamically determine the total PBS walltime
-needed for the pilot job, but this can be specified manually using
-attributes `maxtime` along with `highOverAllocation` and
-`lowOverAllocation`.
-
-`jobThrottle 3.20` specifies that Swift should allow up to 320 apps to
-run on the `raven` site at once. This is typically set to a number
-greater than or equal to the number of slots x compute nodes x apps
-per node (`jobsPerNode` attribute).
-
-`initialscore 10000` is specified to override Swift's automatic
-throttling, and forces an actual throttle value of approximately
-(specifically 1 over) `jobThrottle` * 100 to be used.
-
-The last two attributes specify where and how Swift should perform
-data management. `workdirectory /lus/scratch/{env.USER}/swiftwork`
-specifies where the Swift "application execution sanbox directory"
-used for each app will be located. In some situations this can be a
-directory local to the compute node (eg, for Cray systems, `/dev/shm`
-or `/tmp`, if those are writable by user jobs and the nodes have
-sufficient space in these RAM-based filesystems).
-
-Finally, `stagingMethod sfs` specifies that Swift will copy data to
-and from the shared file system to the application sandbox
-directories.
Deleted: SwiftApps/tryswift/scripts/part5.txt
===================================================================
--- SwiftApps/tryswift/scripts/part5.txt 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/part5.txt 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,112 +0,0 @@
-Part 5: Controlling the compute-node pools where applications run
-=================================================================
-
-In this section we'll use the script `p5.swift`, very similar to
-`p4.swift` of the prior section, to show how we can route apps to
-specific sites, and also how we can make multiple pools of resources
-(on the same or on different computer systems) available to run a
-single Swift script.
-
-////
-image::part05.png[align="center"]
-
-.p5.swift
-----
-sys::[cat ../part05/p5.swift]
-----
-////
-
-First, lets specify that the analysis app `stats.sh` should be run on
-the local login node instead of on the cluster. This is done simply by
-change the site name field of the analyze app in the apps file:
-
------
-$ cat apps
-raven simulate simulate.sh
-localhost stats stats.sh
------
-
-Running this with `swift p5.swift` we see:
------
-$ grep "on node:" output/*.log
-output/average.log:Running on node: raven
-output/sim_0.log:Running on node: nid00029
-output/sim_1.log:Running on node: nid00029
-output/sim_2.log:Running on node: nid00029
-output/sim_3.log:Running on node: nid00029
-output/sim_4.log:Running on node: nid00029
-output/sim_5.log:Running on node: nid00029
-output/sim_6.log:Running on node: nid00029
-output/sim_7.log:Running on node: nid00029
-output/sim_8.log:Running on node: nid00029
-output/sim_9.log:Running on node: nid00029
------
-
-
-Now lets make further use of Swift's ability to route specific apps to
-specific pools of resources. The Cray Raven system has two node types,
-XE6 32-core 2 x IL-16, and XK7 16-core 1 x IL-16 plus one GPU. Each
-"pool" of nodes has different queue characteristics. We can define
-these differences to Swift as two separate pools, and then spread the
-load of executing a large ensemble of simulations across all the
-pools. (And we'll continue to run the analysis script on a third pool,
-comprising the single login host.)
-
-We use the following `apps` file:
------
-$ cat multipools
-raven simulate simulate.sh
-ravenGPU simulate simulate.sh
-localhost stats stats.sh
-----
-and we adjust the sites file to specify 4-node jobs in the Raven pool:
------
- <profile namespace="globus" key="maxNodes">4</profile>
- <profile namespace="globus" key="nodeGranularity">4</profile>
------
-This results in these PBS jobs:
------
-p01532 at raven:~> qstat -u $USER
-
-Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
---------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
-288687.sdb p01532 small B0827-3406 9919 4 128 -- 00:59 R 00:00
-288688.sdb p01532 gpu_node B0827-3406 9931 6 96 -- 00:59 R 00:00
------
-... and achieves the following parallelism (of about 224 concurrent app tasks):
-
------
-$ swift -tc.file multipools p5.swift -nsim=1000 -steps=3
-Swift 0.94.1 RC2 swift-r6895 cog-r3765
-
-RunID: 20130827-1829-o3h6mht5
-Progress: time: Tue, 27 Aug 2013 18:29:31 -0500
-Progress: time: Tue, 27 Aug 2013 18:29:32 -0500 Initializing:997
-Progress: time: Tue, 27 Aug 2013 18:29:34 -0500 Selecting site:774 Submitting:225 Submitted:1
-Progress: time: Tue, 27 Aug 2013 18:29:35 -0500 Selecting site:774 Stage in:1 Submitted:225
-Progress: time: Tue, 27 Aug 2013 18:29:36 -0500 Selecting site:774 Stage in:1 Submitted:37 Active:188
-Progress: time: Tue, 27 Aug 2013 18:29:39 -0500 Selecting site:774 Submitted:2 Active:223 Stage out:1
-Progress: time: Tue, 27 Aug 2013 18:29:40 -0500 Selecting site:750 Submitted:17 Active:208 Stage out:1 Finished successfully:24
-Progress: time: Tue, 27 Aug 2013 18:29:41 -0500 Selecting site:640 Stage in:1 Submitted:51 Active:174 Finished successfully:134
-Progress: time: Tue, 27 Aug 2013 18:29:42 -0500 Selecting site:551 Submitted:11 Active:214 Stage out:1 Finished successfully:223
-Progress: time: Tue, 27 Aug 2013 18:29:43 -0500 Selecting site:542 Submitted:2 Active:223 Stage out:1 Finished successfully:232
-Progress: time: Tue, 27 Aug 2013 18:29:44 -0500 Selecting site:511 Submitting:1 Submitted:19 Active:206 Finished successfully:263
-Progress: time: Tue, 27 Aug 2013 18:29:45 -0500 Selecting site:463 Stage in:1 Submitted:43 Active:182 Finished successfully:311
-Progress: time: Tue, 27 Aug 2013 18:29:46 -0500 Selecting site:367 Submitting:1 Submitted:38 Active:186 Stage out:1 Finished successfully:407
-Progress: time: Tue, 27 Aug 2013 18:29:47 -0500 Selecting site:309 Submitted:2 Active:223 Stage out:1 Finished successfully:465
-Progress: time: Tue, 27 Aug 2013 18:29:48 -0500 Selecting site:300 Submitted:2 Active:223 Stage out:1 Finished successfully:474
-Progress: time: Tue, 27 Aug 2013 18:29:50 -0500 Selecting site:259 Submitted:11 Active:214 Stage out:1 Finished successfully:515
-Progress: time: Tue, 27 Aug 2013 18:29:51 -0500 Selecting site:201 Stage in:1 Submitted:39 Active:186 Finished successfully:573
-Progress: time: Tue, 27 Aug 2013 18:29:52 -0500 Selecting site:80 Submitted:42 Active:184 Finished successfully:694
-Progress: time: Tue, 27 Aug 2013 18:29:53 -0500 Selecting site:54 Submitted:2 Active:223 Stage out:1 Finished successfully:720
-Progress: time: Tue, 27 Aug 2013 18:29:54 -0500 Selecting site:32 Submitted:4 Active:220 Stage out:1 Finished successfully:743
-Progress: time: Tue, 27 Aug 2013 18:29:55 -0500 Submitted:3 Active:216 Stage out:1 Finished successfully:780
-Progress: time: Tue, 27 Aug 2013 18:29:56 -0500 Stage in:1 Active:143 Finished successfully:856
-Progress: time: Tue, 27 Aug 2013 18:29:57 -0500 Active:38 Stage out:1 Finished successfully:961
-Progress: time: Tue, 27 Aug 2013 18:29:58 -0500 Active:8 Stage out:1 Finished successfully:991
-Progress: time: Tue, 27 Aug 2013 18:29:59 -0500 Stage out:1 Finished successfully:999
-Progress: time: Tue, 27 Aug 2013 18:30:01 -0500 Stage in:1 Finished successfully:1000
-Progress: time: Tue, 27 Aug 2013 18:30:02 -0500 Active:1 Finished successfully:1000
-Progress: time: Tue, 27 Aug 2013 18:30:06 -0500 Stage out:1 Finished successfully:1000
-Final status: Tue, 27 Aug 2013 18:30:07 -0500 Finished successfully:1001
------
Deleted: SwiftApps/tryswift/scripts/part6.txt
===================================================================
--- SwiftApps/tryswift/scripts/part6.txt 2013-12-11 19:31:24 UTC (rev 7413)
+++ SwiftApps/tryswift/scripts/part6.txt 2013-12-11 22:22:44 UTC (rev 7414)
@@ -1,134 +0,0 @@
-Part 6: Specifying more complex workflow patterns
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-p6.swift expands the workflow pattern of p4.swift to add additional
-stages to the workflow. Here, we generate a dynamic seed value that
-will be used by all of the simulations, and for each simulation, we
-run an pre-processing application to generate a unique "bias
-file". This pattern is shown below, followed by the Swift script.
-
-image::part06.png[align="center"]
-
-.p6.swift
-----
-sys::[cat ../part06/p6.swift]
-----
-
-Note that the workflow is based on data flow dependencies: each simulation depends on the seed value, calculated in these two dependent statements:
------
-seedfile = genseed(1);
-int seedval = readData(seedfile);
------
-and on the bias file, computed and then consumed in these two dependent statements:
------
- biasfile = genbias(1000, 20);
- (simout,simlog) = simulation(steps, range, biasfile, 1000000, values);
------
-
-To run:
-----
-$ cd ../part06
-$ swift p6.swift
-----
-
-The default parameters result in the following execution log:
-
------
-$ swift p6.swift
-Swift 0.94.1 RC2 swift-r6895 cog-r3765
-
-RunID: 20130827-1917-jvs4gqm5
-Progress: time: Tue, 27 Aug 2013 19:17:56 -0500
-
-*** Script parameters: nsim=10 range=100 num values=10
-
-Progress: time: Tue, 27 Aug 2013 19:17:57 -0500 Stage in:1 Submitted:10
-Generated seed=382537
-Progress: time: Tue, 27 Aug 2013 19:17:59 -0500 Active:9 Stage out:1 Finished successfully:11
-Final status: Tue, 27 Aug 2013 19:18:00 -0500 Finished successfully:22
------
-which produces the following output:
------
-$ ls -lrt output
-total 264
--rw-r--r-- 1 p01532 61532 9 Aug 27 19:17 seed.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_9.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_8.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_7.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_6.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_5.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_4.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_3.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_2.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_1.dat
--rw-r--r-- 1 p01532 61532 180 Aug 27 19:17 bias_0.dat
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_9.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_9.log
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_8.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_7.out
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_6.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_6.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_5.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_5.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:17 sim_4.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_4.log
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:17 sim_1.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_8.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:18 sim_7.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_3.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:18 sim_3.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_2.out
--rw-r--r-- 1 p01532 61532 14898 Aug 27 19:18 sim_2.log
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_1.out
--rw-r--r-- 1 p01532 61532 90 Aug 27 19:18 sim_0.out
--rw-r--r-- 1 p01532 61532 14897 Aug 27 19:18 sim_0.log
--rw-r--r-- 1 p01532 61532 9 Aug 27 19:18 average.out
--rw-r--r-- 1 p01532 61532 14675 Aug 27 19:18 average.log
------
-
-Each sim_N.out file is the sum of its bias file plus newly "simulated" random output scaled by 1,000,000:
-
------
-$ cat output/bias_0.dat
- 302
- 489
- 81
- 582
- 664
- 290
- 839
- 258
- 506
- 310
- 293
- 508
- 88
- 261
- 453
- 187
- 26
- 198
- 402
- 555
-
-$ cat output/sim_0.out
-64000302
-38000489
-32000081
-12000582
-46000664
-36000290
-35000839
-22000258
-49000506
-75000310
------
-
-We produce 20 values in each bias file. Simulations of less than that
-number of values ignore the unneeded number, while simualtions of more
-than 20 will use the last bias number for all remoaining values past
-20. As an exercise, adjust the code to produce the same number of
-bias values as is needed for each simulation. As a further exercise,
-modify the script to generate a unique seed value for each simulation,
-which is a common practice in ensemble computations.
-
More information about the Swift-commit
mailing list