[Swift-commit] r5079 - in trunk/docs: . cookbook documentation quickstart siteguide stylesheets tutorial tutorial/images tutorial/images/icons userguide utils
davidk at ci.uchicago.edu
davidk at ci.uchicago.edu
Wed Sep 7 11:32:34 CDT 2011
Author: davidk
Date: 2011-09-07 11:32:23 -0500 (Wed, 07 Sep 2011)
New Revision: 5079
Added:
trunk/docs/README
trunk/docs/cookbook/cookbook.txt
trunk/docs/documentation/
trunk/docs/documentation/automation
trunk/docs/documentation/building
trunk/docs/documentation/building.html
trunk/docs/documentation/documentation.txt
trunk/docs/documentation/overview
trunk/docs/documentation/structure
trunk/docs/print_link.sh
trunk/docs/quickstart/
trunk/docs/quickstart/quickstart.txt
trunk/docs/siteguide/
trunk/docs/siteguide/beagle
trunk/docs/siteguide/futuregrid
trunk/docs/siteguide/images
trunk/docs/siteguide/intrepid
trunk/docs/siteguide/pads
trunk/docs/siteguide/prereqs
trunk/docs/siteguide/siteguide.txt
trunk/docs/stylesheets/
trunk/docs/stylesheets/asciidoc.css
trunk/docs/tutorial/abstract
trunk/docs/tutorial/bits
trunk/docs/tutorial/hello_world
trunk/docs/tutorial/images/
trunk/docs/tutorial/images/icons/
trunk/docs/tutorial/images/icons/caution.png
trunk/docs/tutorial/images/icons/example.png
trunk/docs/tutorial/images/icons/home.png
trunk/docs/tutorial/images/icons/important.png
trunk/docs/tutorial/images/icons/next.png
trunk/docs/tutorial/images/icons/note.png
trunk/docs/tutorial/images/icons/prev.png
trunk/docs/tutorial/images/icons/tip.png
trunk/docs/tutorial/images/icons/up.png
trunk/docs/tutorial/images/icons/warning.png
trunk/docs/tutorial/introduction
trunk/docs/tutorial/language_features
trunk/docs/tutorial/runtime_features
trunk/docs/tutorial/tutorial.txt
trunk/docs/userguide/log-processing
trunk/docs/utils/
trunk/docs/utils/gensites.txt
Removed:
trunk/docs/README
trunk/docs/cookbook/cookbook-asciidoc.txt
trunk/docs/documentation/automation
trunk/docs/documentation/building
trunk/docs/documentation/building.html
trunk/docs/documentation/documentation.txt
trunk/docs/documentation/overview
trunk/docs/documentation/structure
trunk/docs/print_link.sh
trunk/docs/quickstart/quickstart.txt
trunk/docs/siteguide/
trunk/docs/siteguide/beagle
trunk/docs/siteguide/futuregrid
trunk/docs/siteguide/images
trunk/docs/siteguide/intrepid
trunk/docs/siteguide/pads
trunk/docs/siteguide/prereqs
trunk/docs/siteguide/siteguide.txt
trunk/docs/stylesheets/
trunk/docs/stylesheets/asciidoc.css
trunk/docs/tutorial/abstract
trunk/docs/tutorial/bits
trunk/docs/tutorial/hello_world
trunk/docs/tutorial/images/
trunk/docs/tutorial/images/icons/
trunk/docs/tutorial/images/icons/caution.png
trunk/docs/tutorial/images/icons/example.png
trunk/docs/tutorial/images/icons/home.png
trunk/docs/tutorial/images/icons/important.png
trunk/docs/tutorial/images/icons/next.png
trunk/docs/tutorial/images/icons/note.png
trunk/docs/tutorial/images/icons/prev.png
trunk/docs/tutorial/images/icons/tip.png
trunk/docs/tutorial/images/icons/up.png
trunk/docs/tutorial/images/icons/warning.png
trunk/docs/tutorial/introduction
trunk/docs/tutorial/language_features
trunk/docs/tutorial/runtime_features
trunk/docs/tutorial/tutorial.txt
trunk/docs/utils/
trunk/docs/utils/gensites.txt
Modified:
trunk/docs/
trunk/docs/cookbook/
trunk/docs/tutorial/
trunk/docs/userguide/cdm
trunk/docs/userguide/coasters
trunk/docs/userguide/userguide.txt
Log:
Merge 0.93 docs with trunk
Property changes on: trunk/docs
___________________________________________________________________
Added: svn:mergeinfo
+ /branches/release-0.93/docs:4761-5078
Deleted: trunk/docs/README
===================================================================
--- trunk/docs/README 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/README 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,11 +0,0 @@
-Prerequisites to building documentation:
-
-* asciidoc and a2x (http://www.methods.co.nz/asciidoc)
-* dblatex (http://dblatex.sourceforge.net)
-* source-highlight (http://www.gnu.org/software/src-highlite)
-
-For CI users, compiled versions of these packages can be found in ~davidk.
-
-To build, run:
-
-build_docs.sh <installation_dir>
Copied: trunk/docs/README (from rev 5078, branches/release-0.93/docs/README)
===================================================================
--- trunk/docs/README (rev 0)
+++ trunk/docs/README 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,11 @@
+Prerequisites to building documentation:
+
+* asciidoc and a2x (http://www.methods.co.nz/asciidoc)
+* dblatex (http://dblatex.sourceforge.net)
+* source-highlight (http://www.gnu.org/software/src-highlite)
+
+For CI users, compiled versions of these packages can be found in ~davidk.
+
+To build, run:
+
+build_docs.sh <installation_dir>
Property changes on: trunk/docs/cookbook
___________________________________________________________________
Deleted: svn:ignore
- cookbook-asciidoc.html
cookbook-asciidoc.pdf
Deleted: trunk/docs/cookbook/cookbook-asciidoc.txt
===================================================================
--- trunk/docs/cookbook/cookbook-asciidoc.txt 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/cookbook/cookbook-asciidoc.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,20 +0,0 @@
-// To compile use: asciidoc -a toc -n cookbook-asciidoc.txt
-
-:numbered:
-
-Swift Cookbook
-==============
-Swift team <swift-user at ci.uchicago.edu>
-v0.92, March 2011
-
-include::overview[]
-
-include::swift_basics[]
-
-include::coasters[]
-
-include::diverse_infrastructures[]
-
-include::debugging_swift[]
-
-include::log_processing[]
Copied: trunk/docs/cookbook/cookbook.txt (from rev 5078, branches/release-0.93/docs/cookbook/cookbook.txt)
===================================================================
--- trunk/docs/cookbook/cookbook.txt (rev 0)
+++ trunk/docs/cookbook/cookbook.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,16 @@
+Swift Cookbook
+==============
+Swift team <swift-user at ci.uchicago.edu>
+v0.92, March 2011
+
+include::overview[]
+
+include::swift_basics[]
+
+include::coasters[]
+
+include::diverse_infrastructures[]
+
+include::debugging_swift[]
+
+include::log_processing[]
Deleted: trunk/docs/documentation/automation
===================================================================
--- branches/release-0.93/docs/documentation/automation 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/documentation/automation 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,26 +0,0 @@
-Document Automation
--------------------
-All .txt files in the docs directory get converted to HTML and PDF on a nightly
-basis. These documents then get copied to the Swift web site.
-
-Cron Job
-~~~~~~~~
-The job that processes these documents runs on Bridled at 7pm nightly. The script is
-/home/davidk/docscripts/update_and_build.sh. The script calls svn update on several
-different swift directories in ~davidk, and then builds the the documents. The
-documents should be built for each Swift branch later than 0.93, including trunk. As
-new versions of Swift are created, this script will need to be updated.
-
-Website
--------
-Once the cron job runs, documents can be accessed at:
-
-\http://www.ci.uchicago.edu/swift/guides/release-<version>/documentname/documentname.html
-\http://www.ci.uchicago.edu/swift/guides/release-<version>/documentname/documentname.pdf
-
-For example, assume that you create a document called docs/newdoc. You check this
-guide into Swift 0.93. The AsciiDoc .txt file is called newdoc.txt. The URLs
-that would be generated would be:
-
-\http://www.ci.uchicago.edu/swift/guides/release-0.93/newdoc/newdoc.html
-\http://www.ci.uchicago.edu/swift/guides/release-0.93/newdoc/newdoc.pdf
Copied: trunk/docs/documentation/automation (from rev 5078, branches/release-0.93/docs/documentation/automation)
===================================================================
--- trunk/docs/documentation/automation (rev 0)
+++ trunk/docs/documentation/automation 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,26 @@
+Document Automation
+-------------------
+All .txt files in the docs directory get converted to HTML and PDF on a nightly
+basis. These documents then get copied to the Swift web site.
+
+Cron Job
+~~~~~~~~
+The job that processes these documents runs on Bridled at 7pm nightly. The script is
+/home/davidk/docscripts/update_and_build.sh. The script calls svn update on several
+different swift directories in ~davidk, and then builds the the documents. The
+documents should be built for each Swift branch later than 0.93, including trunk. As
+new versions of Swift are created, this script will need to be updated.
+
+Website
+-------
+Once the cron job runs, documents can be accessed at:
+
+\http://www.ci.uchicago.edu/swift/guides/release-<version>/documentname/documentname.html
+\http://www.ci.uchicago.edu/swift/guides/release-<version>/documentname/documentname.pdf
+
+For example, assume that you create a document called docs/newdoc. You check this
+guide into Swift 0.93. The AsciiDoc .txt file is called newdoc.txt. The URLs
+that would be generated would be:
+
+\http://www.ci.uchicago.edu/swift/guides/release-0.93/newdoc/newdoc.html
+\http://www.ci.uchicago.edu/swift/guides/release-0.93/newdoc/newdoc.pdf
Deleted: trunk/docs/documentation/building
===================================================================
--- branches/release-0.93/docs/documentation/building 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/documentation/building 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,26 +0,0 @@
-Building Documentation Manually
--------------------------------
-Before committing anything to SVN, it is a good idea manually
-build the documentation for testing purposes.
-
-Prerequisites
-~~~~~~~~~~~~~
-There are several prerequisites for building documentation:
-
-* asciidoc and a2x (http://www.methods.co.nz/asciidoc)
-* dblatex (http://dblatex.sourceforge.net)
-* source-highlight (http://www.gnu.org/software/src-highlite)
-
-For CI users, compiled versions of these packages can be found in ~davidk.
-
-build_docs.sh
-~~~~~~~~~~~~~
-
-The script to build the documentation is docs/build_docs.sh. This script
-will convert the documents into HTML and PDF, and then copy the
-files to a given output directory.
-
-.Usage
------
-./build_docs.sh /path/to/copy/output
------
Copied: trunk/docs/documentation/building (from rev 5078, branches/release-0.93/docs/documentation/building)
===================================================================
--- trunk/docs/documentation/building (rev 0)
+++ trunk/docs/documentation/building 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,26 @@
+Building Documentation Manually
+-------------------------------
+Before committing anything to SVN, it is a good idea manually
+build the documentation for testing purposes.
+
+Prerequisites
+~~~~~~~~~~~~~
+There are several prerequisites for building documentation:
+
+* asciidoc and a2x (http://www.methods.co.nz/asciidoc)
+* dblatex (http://dblatex.sourceforge.net)
+* source-highlight (http://www.gnu.org/software/src-highlite)
+
+For CI users, compiled versions of these packages can be found in ~davidk.
+
+build_docs.sh
+~~~~~~~~~~~~~
+
+The script to build the documentation is docs/build_docs.sh. This script
+will convert the documents into HTML and PDF, and then copy the
+files to a given output directory.
+
+.Usage
+-----
+./build_docs.sh /path/to/copy/output
+-----
Deleted: trunk/docs/documentation/building.html
===================================================================
--- branches/release-0.93/docs/documentation/building.html 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/documentation/building.html 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,576 +0,0 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
- "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
-<head>
-<meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
-<meta name="generator" content="AsciiDoc 8.6.3" />
-<title></title>
-<style type="text/css">
-/* Sans-serif font. */
-h1, h2, h3, h4, h5, h6,
-div.title, caption.title,
-thead, p.table.header,
-div#toctitle,
-span#author, span#revnumber, span#revdate, span#revremark,
-div#footer {
- font-family: Arial,Helvetica,sans-serif;
-}
-
-/* Serif font. */
-div.sectionbody {
- font-family: Georgia,"Times New Roman",Times,serif;
-}
-
-/* Monospace font. */
-tt {
- font-size: inherit;
-}
-
-body {
- margin: 1em 5% 1em 5%;
-}
-
-a {
- color: blue;
- text-decoration: underline;
-}
-a:visited {
- color: fuchsia;
-}
-
-em {
- font-style: italic;
- color: navy;
-}
-
-strong {
- font-weight: bold;
- color: #083194;
-}
-
-tt {
- font-size: inherit;
- color: navy;
-}
-
-h1, h2, h3, h4, h5, h6 {
- color: #527bbd;
- margin-top: 1.2em;
- margin-bottom: 0.5em;
- line-height: 1.3;
-}
-
-h1, h2, h3 {
- border-bottom: 2px solid silver;
-}
-h2 {
- padding-top: 0.5em;
-}
-h3 {
- float: left;
-}
-h3 + * {
- clear: left;
-}
-
-div.sectionbody {
- margin-left: 0;
-}
-
-hr {
- border: 1px solid silver;
-}
-
-p {
- margin-top: 0.5em;
- margin-bottom: 0.5em;
-}
-
-ul, ol, li > p {
- margin-top: 0;
-}
-ul > li { color: #aaa; }
-ul > li > * { color: black; }
-
-pre {
- padding: 0;
- margin: 0;
-}
-
-span#author {
- color: #527bbd;
- font-weight: bold;
- font-size: 1.1em;
-}
-span#email {
-}
-span#revnumber, span#revdate, span#revremark {
-}
-
-div#footer {
- font-size: small;
- border-top: 2px solid silver;
- padding-top: 0.5em;
- margin-top: 4.0em;
-}
-div#footer-text {
- float: left;
- padding-bottom: 0.5em;
-}
-div#footer-badges {
- float: right;
- padding-bottom: 0.5em;
-}
-
-div#preamble {
- margin-top: 1.5em;
- margin-bottom: 1.5em;
-}
-div.tableblock, div.imageblock, div.exampleblock, div.verseblock,
-div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
-div.admonitionblock {
- margin-top: 1.0em;
- margin-bottom: 1.5em;
-}
-div.admonitionblock {
- margin-top: 2.0em;
- margin-bottom: 2.0em;
- margin-right: 10%;
- color: #606060;
-}
-
-div.content { /* Block element content. */
- padding: 0;
-}
-
-/* Block element titles. */
-div.title, caption.title {
- color: #527bbd;
- font-weight: bold;
- text-align: left;
- margin-top: 1.0em;
- margin-bottom: 0.5em;
-}
-div.title + * {
- margin-top: 0;
-}
-
-td div.title:first-child {
- margin-top: 0.0em;
-}
-div.content div.title:first-child {
- margin-top: 0.0em;
-}
-div.content + div.title {
- margin-top: 0.0em;
-}
-
-div.sidebarblock > div.content {
- background: #ffffee;
- border: 1px solid #dddddd;
- border-left: 4px solid #f0f0f0;
- padding: 0.5em;
-}
-
-div.listingblock > div.content {
- border: 1px solid #dddddd;
- border-left: 5px solid #f0f0f0;
- background: #f8f8f8;
- padding: 0.5em;
-}
-
-div.quoteblock, div.verseblock {
- padding-left: 1.0em;
- margin-left: 1.0em;
- margin-right: 10%;
- border-left: 5px solid #f0f0f0;
- color: #777777;
-}
-
-div.quoteblock > div.attribution {
- padding-top: 0.5em;
- text-align: right;
-}
-
-div.verseblock > pre.content {
- font-family: inherit;
- font-size: inherit;
-}
-div.verseblock > div.attribution {
- padding-top: 0.75em;
- text-align: left;
-}
-/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
-div.verseblock + div.attribution {
- text-align: left;
-}
-
-div.admonitionblock .icon {
- vertical-align: top;
- font-size: 1.1em;
- font-weight: bold;
- text-decoration: underline;
- color: #527bbd;
- padding-right: 0.5em;
-}
-div.admonitionblock td.content {
- padding-left: 0.5em;
- border-left: 3px solid #dddddd;
-}
-
-div.exampleblock > div.content {
- border-left: 3px solid #dddddd;
- padding-left: 0.5em;
-}
-
-div.imageblock div.content { padding-left: 0; }
-span.image img { border-style: none; }
-a.image:visited { color: white; }
-
-dl {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-dt {
- margin-top: 0.5em;
- margin-bottom: 0;
- font-style: normal;
- color: navy;
-}
-dd > *:first-child {
- margin-top: 0.1em;
-}
-
-ul, ol {
- list-style-position: outside;
-}
-ol.arabic {
- list-style-type: decimal;
-}
-ol.loweralpha {
- list-style-type: lower-alpha;
-}
-ol.upperalpha {
- list-style-type: upper-alpha;
-}
-ol.lowerroman {
- list-style-type: lower-roman;
-}
-ol.upperroman {
- list-style-type: upper-roman;
-}
-
-div.compact ul, div.compact ol,
-div.compact p, div.compact p,
-div.compact div, div.compact div {
- margin-top: 0.1em;
- margin-bottom: 0.1em;
-}
-
-div.tableblock > table {
- border: 3px solid #527bbd;
-}
-thead, p.table.header {
- font-weight: bold;
- color: #527bbd;
-}
-tfoot {
- font-weight: bold;
-}
-td > div.verse {
- white-space: pre;
-}
-p.table {
- margin-top: 0;
-}
-/* Because the table frame attribute is overriden by CSS in most browsers. */
-div.tableblock > table[frame="void"] {
- border-style: none;
-}
-div.tableblock > table[frame="hsides"] {
- border-left-style: none;
- border-right-style: none;
-}
-div.tableblock > table[frame="vsides"] {
- border-top-style: none;
- border-bottom-style: none;
-}
-
-
-div.hdlist {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-div.hdlist tr {
- padding-bottom: 15px;
-}
-dt.hdlist1.strong, td.hdlist1.strong {
- font-weight: bold;
-}
-td.hdlist1 {
- vertical-align: top;
- font-style: normal;
- padding-right: 0.8em;
- color: navy;
-}
-td.hdlist2 {
- vertical-align: top;
-}
-div.hdlist.compact tr {
- margin: 0;
- padding-bottom: 0;
-}
-
-.comment {
- background: yellow;
-}
-
-.footnote, .footnoteref {
- font-size: 0.8em;
-}
-
-span.footnote, span.footnoteref {
- vertical-align: super;
-}
-
-#footnotes {
- margin: 20px 0 20px 0;
- padding: 7px 0 0 0;
-}
-
-#footnotes div.footnote {
- margin: 0 0 5px 0;
-}
-
-#footnotes hr {
- border: none;
- border-top: 1px solid silver;
- height: 1px;
- text-align: left;
- margin-left: 0;
- width: 20%;
- min-width: 100px;
-}
-
-div.colist td {
- padding-right: 0.5em;
- padding-bottom: 0.3em;
- vertical-align: top;
-}
-div.colist td img {
- margin-top: 0.3em;
-}
-
- at media print {
- div#footer-badges { display: none; }
-}
-
-div#toc {
- margin-bottom: 2.5em;
-}
-
-div#toctitle {
- color: #527bbd;
- font-size: 1.1em;
- font-weight: bold;
- margin-top: 1.0em;
- margin-bottom: 0.1em;
-}
-
-div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
- margin-top: 0;
- margin-bottom: 0;
-}
-div.toclevel2 {
- margin-left: 2em;
- font-size: 0.9em;
-}
-div.toclevel3 {
- margin-left: 4em;
- font-size: 0.9em;
-}
-div.toclevel4 {
- margin-left: 6em;
- font-size: 0.9em;
-}
-
-</style>
-<script type="text/javascript">
-/*<+'])');
- // Function that scans the DOM tree for header elements (the DOM2
- // nodeIterator API would be a better technique but not supported by all
- // browsers).
- var iterate = function (el) {
- for (var i = el.firstChild; i != null; i = i.nextSibling) {
- if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
- var mo = re.exec(i.tagName);
- if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
- result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
- }
- iterate(i);
- }
- }
- }
- iterate(el);
- return result;
- }
-
- var toc = document.getElementById("toc");
- var entries = tocEntries(document.getElementById("content"), toclevels);
- for (var i = 0; i < entries.length; ++i) {
- var entry = entries[i];
- if (entry.element.id == "")
- entry.element.id = "_toc_" + i;
- var a = document.createElement("a");
- a.href = "#" + entry.element.id;
- a.appendChild(document.createTextNode(entry.text));
- var div = document.createElement("div");
- div.appendChild(a);
- div.className = "toclevel" + entry.toclevel;
- toc.appendChild(div);
- }
- if (entries.length == 0)
- toc.parentNode.removeChild(toc);
-},
-
-
-/////////////////////////////////////////////////////////////////////
-// Footnotes generator
-/////////////////////////////////////////////////////////////////////
-
-/* Based on footnote generation code from:
- * http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
- */
-
-footnotes: function () {
- var cont = document.getElementById("content");
- var noteholder = document.getElementById("footnotes");
- var spans = cont.getElementsByTagName("span");
- var refs = {};
- var n = 0;
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnote") {
- n++;
- // Use [\s\S] in place of . so multi-line matches work.
- // Because JavaScript has no s (dotall) regex flag.
- note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
- noteholder.innerHTML +=
- "<div class='footnote' id='_footnote_" + n + "'>" +
- "<a href='#_footnoteref_" + n + "' title='Return to text'>" +
- n + "</a>. " + note + "</div>";
- spans[i].innerHTML =
- "[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- var id =spans[i].getAttribute("id");
- if (id != null) refs["#"+id] = n;
- }
- }
- if (n == 0)
- noteholder.parentNode.removeChild(noteholder);
- else {
- // Process footnoterefs.
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnoteref") {
- var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
- href = href.match(/#.*/)[0]; // Because IE return full URL.
- n = refs[href];
- spans[i].innerHTML =
- "[<a href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- }
- }
- }
-}
-
-}
-/*]]>*/
-</script>
-</head>
-<body class="article">
-<div id="header">
-</div>
-<div id="content">
-<div class="sect1">
-<h2 id="_prerequisites">Prerequisites</h2>
-<div class="sectionbody">
-<div class="paragraph"><p>There are several prerequisites for building documentation:</p></div>
-<div class="ulist"><ul>
-<li>
-<p>
-asciidoc and a2x (<a href="http://www.methods.co.nz/asciidoc">http://www.methods.co.nz/asciidoc</a>)
-</p>
-</li>
-<li>
-<p>
-dblatex (<a href="http://dblatex.sourceforge.net">http://dblatex.sourceforge.net</a>)
-</p>
-</li>
-<li>
-<p>
-source-highlight (<a href="http://www.gnu.org/software/src-highlite">http://www.gnu.org/software/src-highlite</a>)
-</p>
-</li>
-</ul></div>
-<div class="paragraph"><p>For CI users, compiled versions of these packages can be found in ~davidk.</p></div>
-<div class="sect2">
-<h3 id="_build_docs_script">Build Docs Script</h3>
-<div class="paragraph"><p>Before committing any changes to documentation, it is a good idea manually
-build the documentation. The script to build the documentation is
-docs/build_docs.sh.</p></div>
-<div class="paragraph"><p>This script will convert the documents into HTML and PDF, and then copy the
-files to a given output directory.</p></div>
-</div>
-</div>
-</div>
-</div>
-<div id="footnotes"><hr /></div>
-<div id="footer">
-<div id="footer-text">
-Last updated 2011-08-31 12:17:55 CDT
-</div>
-</div>
-</body>
-</html>
Copied: trunk/docs/documentation/building.html (from rev 5078, branches/release-0.93/docs/documentation/building.html)
===================================================================
--- trunk/docs/documentation/building.html (rev 0)
+++ trunk/docs/documentation/building.html 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,576 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
+ "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
+<head>
+<meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
+<meta name="generator" content="AsciiDoc 8.6.3" />
+<title></title>
+<style type="text/css">
+/* Sans-serif font. */
+h1, h2, h3, h4, h5, h6,
+div.title, caption.title,
+thead, p.table.header,
+div#toctitle,
+span#author, span#revnumber, span#revdate, span#revremark,
+div#footer {
+ font-family: Arial,Helvetica,sans-serif;
+}
+
+/* Serif font. */
+div.sectionbody {
+ font-family: Georgia,"Times New Roman",Times,serif;
+}
+
+/* Monospace font. */
+tt {
+ font-size: inherit;
+}
+
+body {
+ margin: 1em 5% 1em 5%;
+}
+
+a {
+ color: blue;
+ text-decoration: underline;
+}
+a:visited {
+ color: fuchsia;
+}
+
+em {
+ font-style: italic;
+ color: navy;
+}
+
+strong {
+ font-weight: bold;
+ color: #083194;
+}
+
+tt {
+ font-size: inherit;
+ color: navy;
+}
+
+h1, h2, h3, h4, h5, h6 {
+ color: #527bbd;
+ margin-top: 1.2em;
+ margin-bottom: 0.5em;
+ line-height: 1.3;
+}
+
+h1, h2, h3 {
+ border-bottom: 2px solid silver;
+}
+h2 {
+ padding-top: 0.5em;
+}
+h3 {
+ float: left;
+}
+h3 + * {
+ clear: left;
+}
+
+div.sectionbody {
+ margin-left: 0;
+}
+
+hr {
+ border: 1px solid silver;
+}
+
+p {
+ margin-top: 0.5em;
+ margin-bottom: 0.5em;
+}
+
+ul, ol, li > p {
+ margin-top: 0;
+}
+ul > li { color: #aaa; }
+ul > li > * { color: black; }
+
+pre {
+ padding: 0;
+ margin: 0;
+}
+
+span#author {
+ color: #527bbd;
+ font-weight: bold;
+ font-size: 1.1em;
+}
+span#email {
+}
+span#revnumber, span#revdate, span#revremark {
+}
+
+div#footer {
+ font-size: small;
+ border-top: 2px solid silver;
+ padding-top: 0.5em;
+ margin-top: 4.0em;
+}
+div#footer-text {
+ float: left;
+ padding-bottom: 0.5em;
+}
+div#footer-badges {
+ float: right;
+ padding-bottom: 0.5em;
+}
+
+div#preamble {
+ margin-top: 1.5em;
+ margin-bottom: 1.5em;
+}
+div.tableblock, div.imageblock, div.exampleblock, div.verseblock,
+div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
+div.admonitionblock {
+ margin-top: 1.0em;
+ margin-bottom: 1.5em;
+}
+div.admonitionblock {
+ margin-top: 2.0em;
+ margin-bottom: 2.0em;
+ margin-right: 10%;
+ color: #606060;
+}
+
+div.content { /* Block element content. */
+ padding: 0;
+}
+
+/* Block element titles. */
+div.title, caption.title {
+ color: #527bbd;
+ font-weight: bold;
+ text-align: left;
+ margin-top: 1.0em;
+ margin-bottom: 0.5em;
+}
+div.title + * {
+ margin-top: 0;
+}
+
+td div.title:first-child {
+ margin-top: 0.0em;
+}
+div.content div.title:first-child {
+ margin-top: 0.0em;
+}
+div.content + div.title {
+ margin-top: 0.0em;
+}
+
+div.sidebarblock > div.content {
+ background: #ffffee;
+ border: 1px solid #dddddd;
+ border-left: 4px solid #f0f0f0;
+ padding: 0.5em;
+}
+
+div.listingblock > div.content {
+ border: 1px solid #dddddd;
+ border-left: 5px solid #f0f0f0;
+ background: #f8f8f8;
+ padding: 0.5em;
+}
+
+div.quoteblock, div.verseblock {
+ padding-left: 1.0em;
+ margin-left: 1.0em;
+ margin-right: 10%;
+ border-left: 5px solid #f0f0f0;
+ color: #777777;
+}
+
+div.quoteblock > div.attribution {
+ padding-top: 0.5em;
+ text-align: right;
+}
+
+div.verseblock > pre.content {
+ font-family: inherit;
+ font-size: inherit;
+}
+div.verseblock > div.attribution {
+ padding-top: 0.75em;
+ text-align: left;
+}
+/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
+div.verseblock + div.attribution {
+ text-align: left;
+}
+
+div.admonitionblock .icon {
+ vertical-align: top;
+ font-size: 1.1em;
+ font-weight: bold;
+ text-decoration: underline;
+ color: #527bbd;
+ padding-right: 0.5em;
+}
+div.admonitionblock td.content {
+ padding-left: 0.5em;
+ border-left: 3px solid #dddddd;
+}
+
+div.exampleblock > div.content {
+ border-left: 3px solid #dddddd;
+ padding-left: 0.5em;
+}
+
+div.imageblock div.content { padding-left: 0; }
+span.image img { border-style: none; }
+a.image:visited { color: white; }
+
+dl {
+ margin-top: 0.8em;
+ margin-bottom: 0.8em;
+}
+dt {
+ margin-top: 0.5em;
+ margin-bottom: 0;
+ font-style: normal;
+ color: navy;
+}
+dd > *:first-child {
+ margin-top: 0.1em;
+}
+
+ul, ol {
+ list-style-position: outside;
+}
+ol.arabic {
+ list-style-type: decimal;
+}
+ol.loweralpha {
+ list-style-type: lower-alpha;
+}
+ol.upperalpha {
+ list-style-type: upper-alpha;
+}
+ol.lowerroman {
+ list-style-type: lower-roman;
+}
+ol.upperroman {
+ list-style-type: upper-roman;
+}
+
+div.compact ul, div.compact ol,
+div.compact p, div.compact p,
+div.compact div, div.compact div {
+ margin-top: 0.1em;
+ margin-bottom: 0.1em;
+}
+
+div.tableblock > table {
+ border: 3px solid #527bbd;
+}
+thead, p.table.header {
+ font-weight: bold;
+ color: #527bbd;
+}
+tfoot {
+ font-weight: bold;
+}
+td > div.verse {
+ white-space: pre;
+}
+p.table {
+ margin-top: 0;
+}
+/* Because the table frame attribute is overriden by CSS in most browsers. */
+div.tableblock > table[frame="void"] {
+ border-style: none;
+}
+div.tableblock > table[frame="hsides"] {
+ border-left-style: none;
+ border-right-style: none;
+}
+div.tableblock > table[frame="vsides"] {
+ border-top-style: none;
+ border-bottom-style: none;
+}
+
+
+div.hdlist {
+ margin-top: 0.8em;
+ margin-bottom: 0.8em;
+}
+div.hdlist tr {
+ padding-bottom: 15px;
+}
+dt.hdlist1.strong, td.hdlist1.strong {
+ font-weight: bold;
+}
+td.hdlist1 {
+ vertical-align: top;
+ font-style: normal;
+ padding-right: 0.8em;
+ color: navy;
+}
+td.hdlist2 {
+ vertical-align: top;
+}
+div.hdlist.compact tr {
+ margin: 0;
+ padding-bottom: 0;
+}
+
+.comment {
+ background: yellow;
+}
+
+.footnote, .footnoteref {
+ font-size: 0.8em;
+}
+
+span.footnote, span.footnoteref {
+ vertical-align: super;
+}
+
+#footnotes {
+ margin: 20px 0 20px 0;
+ padding: 7px 0 0 0;
+}
+
+#footnotes div.footnote {
+ margin: 0 0 5px 0;
+}
+
+#footnotes hr {
+ border: none;
+ border-top: 1px solid silver;
+ height: 1px;
+ text-align: left;
+ margin-left: 0;
+ width: 20%;
+ min-width: 100px;
+}
+
+div.colist td {
+ padding-right: 0.5em;
+ padding-bottom: 0.3em;
+ vertical-align: top;
+}
+div.colist td img {
+ margin-top: 0.3em;
+}
+
+ at media print {
+ div#footer-badges { display: none; }
+}
+
+div#toc {
+ margin-bottom: 2.5em;
+}
+
+div#toctitle {
+ color: #527bbd;
+ font-size: 1.1em;
+ font-weight: bold;
+ margin-top: 1.0em;
+ margin-bottom: 0.1em;
+}
+
+div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
+ margin-top: 0;
+ margin-bottom: 0;
+}
+div.toclevel2 {
+ margin-left: 2em;
+ font-size: 0.9em;
+}
+div.toclevel3 {
+ margin-left: 4em;
+ font-size: 0.9em;
+}
+div.toclevel4 {
+ margin-left: 6em;
+ font-size: 0.9em;
+}
+
+</style>
+<script type="text/javascript">
+/*<+'])');
+ // Function that scans the DOM tree for header elements (the DOM2
+ // nodeIterator API would be a better technique but not supported by all
+ // browsers).
+ var iterate = function (el) {
+ for (var i = el.firstChild; i != null; i = i.nextSibling) {
+ if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
+ var mo = re.exec(i.tagName);
+ if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
+ result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
+ }
+ iterate(i);
+ }
+ }
+ }
+ iterate(el);
+ return result;
+ }
+
+ var toc = document.getElementById("toc");
+ var entries = tocEntries(document.getElementById("content"), toclevels);
+ for (var i = 0; i < entries.length; ++i) {
+ var entry = entries[i];
+ if (entry.element.id == "")
+ entry.element.id = "_toc_" + i;
+ var a = document.createElement("a");
+ a.href = "#" + entry.element.id;
+ a.appendChild(document.createTextNode(entry.text));
+ var div = document.createElement("div");
+ div.appendChild(a);
+ div.className = "toclevel" + entry.toclevel;
+ toc.appendChild(div);
+ }
+ if (entries.length == 0)
+ toc.parentNode.removeChild(toc);
+},
+
+
+/////////////////////////////////////////////////////////////////////
+// Footnotes generator
+/////////////////////////////////////////////////////////////////////
+
+/* Based on footnote generation code from:
+ * http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
+ */
+
+footnotes: function () {
+ var cont = document.getElementById("content");
+ var noteholder = document.getElementById("footnotes");
+ var spans = cont.getElementsByTagName("span");
+ var refs = {};
+ var n = 0;
+ for (i=0; i<spans.length; i++) {
+ if (spans[i].className == "footnote") {
+ n++;
+ // Use [\s\S] in place of . so multi-line matches work.
+ // Because JavaScript has no s (dotall) regex flag.
+ note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
+ noteholder.innerHTML +=
+ "<div class='footnote' id='_footnote_" + n + "'>" +
+ "<a href='#_footnoteref_" + n + "' title='Return to text'>" +
+ n + "</a>. " + note + "</div>";
+ spans[i].innerHTML =
+ "[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
+ "' title='View footnote' class='footnote'>" + n + "</a>]";
+ var id =spans[i].getAttribute("id");
+ if (id != null) refs["#"+id] = n;
+ }
+ }
+ if (n == 0)
+ noteholder.parentNode.removeChild(noteholder);
+ else {
+ // Process footnoterefs.
+ for (i=0; i<spans.length; i++) {
+ if (spans[i].className == "footnoteref") {
+ var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
+ href = href.match(/#.*/)[0]; // Because IE return full URL.
+ n = refs[href];
+ spans[i].innerHTML =
+ "[<a href='#_footnote_" + n +
+ "' title='View footnote' class='footnote'>" + n + "</a>]";
+ }
+ }
+ }
+}
+
+}
+/*]]>*/
+</script>
+</head>
+<body class="article">
+<div id="header">
+</div>
+<div id="content">
+<div class="sect1">
+<h2 id="_prerequisites">Prerequisites</h2>
+<div class="sectionbody">
+<div class="paragraph"><p>There are several prerequisites for building documentation:</p></div>
+<div class="ulist"><ul>
+<li>
+<p>
+asciidoc and a2x (<a href="http://www.methods.co.nz/asciidoc">http://www.methods.co.nz/asciidoc</a>)
+</p>
+</li>
+<li>
+<p>
+dblatex (<a href="http://dblatex.sourceforge.net">http://dblatex.sourceforge.net</a>)
+</p>
+</li>
+<li>
+<p>
+source-highlight (<a href="http://www.gnu.org/software/src-highlite">http://www.gnu.org/software/src-highlite</a>)
+</p>
+</li>
+</ul></div>
+<div class="paragraph"><p>For CI users, compiled versions of these packages can be found in ~davidk.</p></div>
+<div class="sect2">
+<h3 id="_build_docs_script">Build Docs Script</h3>
+<div class="paragraph"><p>Before committing any changes to documentation, it is a good idea manually
+build the documentation. The script to build the documentation is
+docs/build_docs.sh.</p></div>
+<div class="paragraph"><p>This script will convert the documents into HTML and PDF, and then copy the
+files to a given output directory.</p></div>
+</div>
+</div>
+</div>
+</div>
+<div id="footnotes"><hr /></div>
+<div id="footer">
+<div id="footer-text">
+Last updated 2011-08-31 12:17:55 CDT
+</div>
+</div>
+</body>
+</html>
Deleted: trunk/docs/documentation/documentation.txt
===================================================================
--- branches/release-0.93/docs/documentation/documentation.txt 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/documentation/documentation.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,11 +0,0 @@
-Swift Document Generation
-=========================
-
-include::overview[]
-
-include::structure[]
-
-include::building[]
-
-include::automation[]
-
Copied: trunk/docs/documentation/documentation.txt (from rev 5078, branches/release-0.93/docs/documentation/documentation.txt)
===================================================================
--- trunk/docs/documentation/documentation.txt (rev 0)
+++ trunk/docs/documentation/documentation.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,11 @@
+Swift Document Generation
+=========================
+
+include::overview[]
+
+include::structure[]
+
+include::building[]
+
+include::automation[]
+
Deleted: trunk/docs/documentation/overview
===================================================================
--- branches/release-0.93/docs/documentation/overview 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/documentation/overview 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,5 +0,0 @@
-Overview
---------
-This guide provides a brief overview of how Swift Document Generation works. It
-should explain the structure of a docs directory, how to manually build documentation,
-and explain how document automation works.
Copied: trunk/docs/documentation/overview (from rev 5078, branches/release-0.93/docs/documentation/overview)
===================================================================
--- trunk/docs/documentation/overview (rev 0)
+++ trunk/docs/documentation/overview 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,5 @@
+Overview
+--------
+This guide provides a brief overview of how Swift Document Generation works. It
+should explain the structure of a docs directory, how to manually build documentation,
+and explain how document automation works.
Deleted: trunk/docs/documentation/structure
===================================================================
--- branches/release-0.93/docs/documentation/structure 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/documentation/structure 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,60 +0,0 @@
-Structure
----------
-A specific file and directory structure is required for each document in order to ease the
-process of automation, conversion, and copying.
-
-Directories
-~~~~~~~~~~~
-Each subdirectory within docs/ should correspond to the name of the document.
-This name will also later be used in the URL used to point to the
-document.
-
-.Example Directory Names
------
-docs/userguide
-docs/tutorial
------
-
-AsciiDoc File
-~~~~~~~~~~~~~
-Every document directory should contain one file with a .txt
-extension. This .txt file is the AsciiDoc document that later gets converted
-to various formats.
-
-.Example AsciiDoc Documents
------
-docs/userguide/userguide.txt
-docs/tutorial/tutorial.txt
------
-
-Chapters
-~~~~~~~~
-Each document directory should contain individual chapters
-that make up the document. These chapters should not have any
-file extensions.
-
-.Example Chapters
------
-docs/userguide/overview
-docs/userguide/language
-docs/userguide/mappers
-docs/userguide/commands
------
-
-Including Chapters in a Document
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The .txt file should then include the individual chapters
-with the AsciiDoc include command.
-
-.userguide.txt
------
-\include::overview[]
-
-\include::language[]
-
-\include::mappers[]
-
-\include::commands[]
------
-
-WARNING: Be sure to include spaces between includes. Putting them together with no spaces can cause formatting/compilation errors.
Copied: trunk/docs/documentation/structure (from rev 5078, branches/release-0.93/docs/documentation/structure)
===================================================================
--- trunk/docs/documentation/structure (rev 0)
+++ trunk/docs/documentation/structure 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,60 @@
+Structure
+---------
+A specific file and directory structure is required for each document in order to ease the
+process of automation, conversion, and copying.
+
+Directories
+~~~~~~~~~~~
+Each subdirectory within docs/ should correspond to the name of the document.
+This name will also later be used in the URL used to point to the
+document.
+
+.Example Directory Names
+-----
+docs/userguide
+docs/tutorial
+-----
+
+AsciiDoc File
+~~~~~~~~~~~~~
+Every document directory should contain one file with a .txt
+extension. This .txt file is the AsciiDoc document that later gets converted
+to various formats.
+
+.Example AsciiDoc Documents
+-----
+docs/userguide/userguide.txt
+docs/tutorial/tutorial.txt
+-----
+
+Chapters
+~~~~~~~~
+Each document directory should contain individual chapters
+that make up the document. These chapters should not have any
+file extensions.
+
+.Example Chapters
+-----
+docs/userguide/overview
+docs/userguide/language
+docs/userguide/mappers
+docs/userguide/commands
+-----
+
+Including Chapters in a Document
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The .txt file should then include the individual chapters
+with the AsciiDoc include command.
+
+.userguide.txt
+-----
+\include::overview[]
+
+\include::language[]
+
+\include::mappers[]
+
+\include::commands[]
+-----
+
+WARNING: Be sure to include spaces between includes. Putting them together with no spaces can cause formatting/compilation errors.
Deleted: trunk/docs/print_link.sh
===================================================================
--- trunk/docs/print_link.sh 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/print_link.sh 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,21 +0,0 @@
-#!/bin/bash
-
-# This script is referenced by asciidoc to point to the correct versions of documentation based on version information
-# print_info.sh <guide>
-# This will print the correct URL based on SVN version info
-
-pushd ../.. > /dev/null 2>&1
-VERSION=`svn info |grep URL|awk -F / '{print $NF}'`
-popd > /dev/null 2>&1
-
-# Parse command line arguments
-case "$1" in
- userguide)
- echo http://www.ci.uchicago.edu/swift/guides/$VERSION/userguide/userguide.html[Swift User Guide]
- ;;
- tutorial)
- echo http://www.ci.uchicago.edu/swift/guides/$VERSION/tutorial/tutorial.html[Swift Tutorial]
- ;;
- *) echo http://www.ci.uchicago.edu/swift/docs/index.php[Swift Documentation]
- ;;
-esac
Copied: trunk/docs/print_link.sh (from rev 5078, branches/release-0.93/docs/print_link.sh)
===================================================================
--- trunk/docs/print_link.sh (rev 0)
+++ trunk/docs/print_link.sh 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# This script is referenced by asciidoc to point to the correct versions of documentation based on version information
+# print_info.sh <guide>
+# This will print the correct URL based on SVN version info
+
+pushd ../.. > /dev/null 2>&1
+VERSION=`svn info |grep URL|awk -F / '{print $NF}'`
+popd > /dev/null 2>&1
+
+# Parse command line arguments
+case "$1" in
+ userguide)
+ echo http://www.ci.uchicago.edu/swift/guides/$VERSION/userguide/userguide.html[Swift User Guide]
+ ;;
+ tutorial)
+ echo http://www.ci.uchicago.edu/swift/guides/$VERSION/tutorial/tutorial.html[Swift Tutorial]
+ ;;
+ *) echo http://www.ci.uchicago.edu/swift/docs/index.php[Swift Documentation]
+ ;;
+esac
Deleted: trunk/docs/quickstart/quickstart.txt
===================================================================
--- branches/release-0.93/docs/quickstart/quickstart.txt 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/quickstart/quickstart.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,101 +0,0 @@
-Swift Quickstart
-================
-
-[abstract]
-Abstract
---------
-This guide describes the steps needed to download, install, configure,
-and run the basic examples for Swift. If you are using a pre-installed
-version of Swift, you can skip directly to the configuration section.
-
-Stable Releases vs. Development Releases
-----------------------------------------
-Stable releases of Swift have undergone more extensive testing than development releases.
-In general, they are more stable, have fewer bugs, and have been tested on a variety of
-systems.
-
-The development version of Swift is aimed at developers and testers. The development
-code has the highest chance of containing buggy and untested code. If you need stability
-please use the latest stable release.
-
-Downloading a Swift Distribution
---------------------------------
-There are two main ways of getting the Swift implementation: binary
-releases and the source repository.
-
-Binary Releases
-~~~~~~~~~~~~~~~
-For the majority of users, downloading and installing binary releases is recommended.
-Since Swift is written in Java, the binary packages will run on all supported platforms with
-Java Runtime Environment 1.5 or greater. Binary releases can be obtained from the
-http://www.ci.uchicago.edu/swift/downloads/index.php[Swift downloads page].
-
-Once downloaded, simply unpack the downloaded package (swift-<version>.tar.gz) into a
-directory of your choice:
-
------
-tar -xzvf swift-<version>.tar.gz
------
-
-This will create a swift-<version> directory containing the build.
-
-Source Repository
-~~~~~~~~~~~~~~~~~
-The source code for Swift is available to developers who have an interest in contributing
-new features. To build Swift from source code, you will need http://ant.apache.org/[Apache Ant]
-and http://www.oracle.com/technetwork/java/javase/downloads/index.html[Java JDK]. Once
-built, the dist/swift-svn directory will contain your build.
-
-To download and build Swift 0.93, follow these instructions:
------
-$ mkdir swift-0.93
-$ cd swift-0.93
-$ svn co https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/4.1.9/src/cog
-$ cd cog/modules
-$ svn co https://svn.ci.uchicago.edu/svn/vdl2/branches/release-0.93 swift
-$ cd swift
-$ ant redist
------
-
-Setting your PATH
------------------
-Once Swift is installed, it is useful to add the swift binary to your PATH
-environment variable. To do this, first determine where the Swift bin
-directory is located. If you installed Swift from a binary release, it will
-be in the swift-0.93/bin directory where you installed it. If you followed
-the instructions above for installing Swift from a source repository, it
-will be located in swift-0.93/cog/modules/swift/dist/swift-svn/bin.
-
-Add the following line to the bottom of ~/.bashrc:
-
------
-export PATH=$PATH:/full/path/to/swift
------
-
-When you login, test this out by typing the command
------
-$ which swift
------
-
-This should point you to the path of the Swift binary.
-
-Running Swift Examples
-----------------------
-The Swift examples can be found in the examples directory in the Swift
-distribution. The examples are written in the SwiftScript language, and
-have .swift as a file extension.
-
-Execution of a Swift workflow is done using the swift command, which
-takes the Swift workflow file name as an argument:
-
------
-cd examples/tutorial
-swift hello.swift
------
-
-When you run this application, it should create a file called hello.txt.
-If this file gets created, you have successfully ran your first
-Swift script!
-
-More documentation on how to run Swift can be found at
-http://www.ci.uchicago.edu/swift/docs/index.php,
Copied: trunk/docs/quickstart/quickstart.txt (from rev 5078, branches/release-0.93/docs/quickstart/quickstart.txt)
===================================================================
--- trunk/docs/quickstart/quickstart.txt (rev 0)
+++ trunk/docs/quickstart/quickstart.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,101 @@
+Swift Quickstart
+================
+
+[abstract]
+Abstract
+--------
+This guide describes the steps needed to download, install, configure,
+and run the basic examples for Swift. If you are using a pre-installed
+version of Swift, you can skip directly to the configuration section.
+
+Stable Releases vs. Development Releases
+----------------------------------------
+Stable releases of Swift have undergone more extensive testing than development releases.
+In general, they are more stable, have fewer bugs, and have been tested on a variety of
+systems.
+
+The development version of Swift is aimed at developers and testers. The development
+code has the highest chance of containing buggy and untested code. If you need stability
+please use the latest stable release.
+
+Downloading a Swift Distribution
+--------------------------------
+There are two main ways of getting the Swift implementation: binary
+releases and the source repository.
+
+Binary Releases
+~~~~~~~~~~~~~~~
+For the majority of users, downloading and installing binary releases is recommended.
+Since Swift is written in Java, the binary packages will run on all supported platforms with
+Java Runtime Environment 1.5 or greater. Binary releases can be obtained from the
+http://www.ci.uchicago.edu/swift/downloads/index.php[Swift downloads page].
+
+Once downloaded, simply unpack the downloaded package (swift-<version>.tar.gz) into a
+directory of your choice:
+
+-----
+tar -xzvf swift-<version>.tar.gz
+-----
+
+This will create a swift-<version> directory containing the build.
+
+Source Repository
+~~~~~~~~~~~~~~~~~
+The source code for Swift is available to developers who have an interest in contributing
+new features. To build Swift from source code, you will need http://ant.apache.org/[Apache Ant]
+and http://www.oracle.com/technetwork/java/javase/downloads/index.html[Java JDK]. Once
+built, the dist/swift-svn directory will contain your build.
+
+To download and build Swift 0.93, follow these instructions:
+-----
+$ mkdir swift-0.93
+$ cd swift-0.93
+$ svn co https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/4.1.9/src/cog
+$ cd cog/modules
+$ svn co https://svn.ci.uchicago.edu/svn/vdl2/branches/release-0.93 swift
+$ cd swift
+$ ant redist
+-----
+
+Setting your PATH
+-----------------
+Once Swift is installed, it is useful to add the swift binary to your PATH
+environment variable. To do this, first determine where the Swift bin
+directory is located. If you installed Swift from a binary release, it will
+be in the swift-0.93/bin directory where you installed it. If you followed
+the instructions above for installing Swift from a source repository, it
+will be located in swift-0.93/cog/modules/swift/dist/swift-svn/bin.
+
+Add the following line to the bottom of ~/.bashrc:
+
+-----
+export PATH=$PATH:/full/path/to/swift
+-----
+
+When you login, test this out by typing the command
+-----
+$ which swift
+-----
+
+This should point you to the path of the Swift binary.
+
+Running Swift Examples
+----------------------
+The Swift examples can be found in the examples directory in the Swift
+distribution. The examples are written in the SwiftScript language, and
+have .swift as a file extension.
+
+Execution of a Swift workflow is done using the swift command, which
+takes the Swift workflow file name as an argument:
+
+-----
+cd examples/tutorial
+swift hello.swift
+-----
+
+When you run this application, it should create a file called hello.txt.
+If this file gets created, you have successfully ran your first
+Swift script!
+
+More documentation on how to run Swift can be found at
+http://www.ci.uchicago.edu/swift/docs/index.php,
Deleted: trunk/docs/siteguide/beagle
===================================================================
--- branches/release-0.93/docs/siteguide/beagle 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/siteguide/beagle 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,49 +0,0 @@
-Beagle
-------
-
-Requesting Access
-~~~~~~~~~~~~~~~~~
-If you do not already have a Computation Institute account, you can request
-one at https://www.ci.uchicago.edu/accounts/. This page will give you a list
-of resources you can request access to.
-You already have an existing CI account, but do not have access to Beagle,
-send an email to support at ci.uchicago.edu to request access.
-
-Connecting to a login node
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-Once you have account, you should be able to access a Beagle login
-node with the following command:
-
------
-ssh yourusername at login.beagle.ci.uchicago.edu
------
-
-Follow the steps outlined below to get started with Swift on Beagle:
-
-*step 1.* Load the Swift module on Beagle as follows: +module load swift+
-
-*step 2.* Create and change to a directory where your Swift related work will
-stay. (say, +mkdir swift-lab+, followed by, +cd swift-lab+)
-
-*step 3.* To get started with a simple example running +/bin/cat+ to read an
-input file +data.txt+ and write to an output file +f.nnn.out+, copy the folder
-at +/home/ketan/catsn+ to the above directory. (+cp -r /home/ketan/catsn
-.+ followed by +cd catsn+).
-
-*step 4.* In the sites file: +beagle-coaster.xml+, make the following two
-changes: *1)* change the path of +workdirectory+ to your preferred location
-(say to +/lustre/beagle/$USER/swift-lab/swift.workdir+) and *2)* Change the
-project name to your project (+CI-CCR000013+) . The workdirectory will contain
-execution data related to each run, e.g. wrapper scripts, system information,
-inputs and outputs.
-
-*step 5.* Run the example using following commandline (also found in run.sh):
-+swift -config cf -tc.file tc -sites.file beagle-coaster.xml catsn.swift -n=1+
-. You can further change the value of +-n+ to any arbitrary number to run that
-many number of concurrent +cat+
-
-*step 6.* Check the output in the generated +outdir+ directory (+ls outdir+)
-
-Note: Running from sandbox node or requesting 1 hour walltime for upto 3 nodes
-will get fast prioritized execution. Good for small tests.
-
Copied: trunk/docs/siteguide/beagle (from rev 5078, branches/release-0.93/docs/siteguide/beagle)
===================================================================
--- trunk/docs/siteguide/beagle (rev 0)
+++ trunk/docs/siteguide/beagle 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,49 @@
+Beagle
+------
+
+Requesting Access
+~~~~~~~~~~~~~~~~~
+If you do not already have a Computation Institute account, you can request
+one at https://www.ci.uchicago.edu/accounts/. This page will give you a list
+of resources you can request access to.
+You already have an existing CI account, but do not have access to Beagle,
+send an email to support at ci.uchicago.edu to request access.
+
+Connecting to a login node
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+Once you have account, you should be able to access a Beagle login
+node with the following command:
+
+-----
+ssh yourusername at login.beagle.ci.uchicago.edu
+-----
+
+Follow the steps outlined below to get started with Swift on Beagle:
+
+*step 1.* Load the Swift module on Beagle as follows: +module load swift+
+
+*step 2.* Create and change to a directory where your Swift related work will
+stay. (say, +mkdir swift-lab+, followed by, +cd swift-lab+)
+
+*step 3.* To get started with a simple example running +/bin/cat+ to read an
+input file +data.txt+ and write to an output file +f.nnn.out+, copy the folder
+at +/home/ketan/catsn+ to the above directory. (+cp -r /home/ketan/catsn
+.+ followed by +cd catsn+).
+
+*step 4.* In the sites file: +beagle-coaster.xml+, make the following two
+changes: *1)* change the path of +workdirectory+ to your preferred location
+(say to +/lustre/beagle/$USER/swift-lab/swift.workdir+) and *2)* Change the
+project name to your project (+CI-CCR000013+) . The workdirectory will contain
+execution data related to each run, e.g. wrapper scripts, system information,
+inputs and outputs.
+
+*step 5.* Run the example using following commandline (also found in run.sh):
++swift -config cf -tc.file tc -sites.file beagle-coaster.xml catsn.swift -n=1+
+. You can further change the value of +-n+ to any arbitrary number to run that
+many number of concurrent +cat+
+
+*step 6.* Check the output in the generated +outdir+ directory (+ls outdir+)
+
+Note: Running from sandbox node or requesting 1 hour walltime for upto 3 nodes
+will get fast prioritized execution. Good for small tests.
+
Deleted: trunk/docs/siteguide/futuregrid
===================================================================
--- branches/release-0.93/docs/siteguide/futuregrid 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/siteguide/futuregrid 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,134 +0,0 @@
-Futuregrid Quickstart Guide
----------------------------
-
-Downloading and Building Swift
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The most recent versions of Swift can be found at
-http://www.ci.uchicago.edu/swift/downloads/index.php. To run on futuregrid,
-you will need to download the development/trunk version from SVN.
-
-Adding Swift to your PATH
-~~~~~~~~~~~~~~~~~~~~~~~~~
-Once you have installed Swift, add the Swift binary to your PATH so you can
-easily run it from any directory.
-
-In your home directory, edit the file ".bashrc".
-
-If you have installed Swift via a source repository, add the following line
-at the bottom of .bashrc.
-
------
-export PATH=$PATH:$HOME/cog/modules/swift/dist/swift-svn/bin
------
-
-Requesting Futuregrid Access
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-If you do not already have a futuregrid account, you can follow the
-instructions at https://portal.futuregrid.org/gettingstarted to get started.
-This page provides information on how to create an account, how to join
-a project, how to set up your SSH keys, and how to create a new project.
-
-Downloading Swift VM Tools
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-A set of scripts based around cloudinitd are used to easily start virtual
-machines. To download, change to your home directory and run the
-following command:
-
------
-$ svn co https://svn.ci.uchicago.edu/svn/vdl2/usertools/swift-vm-boot
------
-
-Download your Credentials
-~~~~~~~~~~~~~~~~~~~~~~~~~
-Run the following commands to retrieve your credentials:
------
-$ scp yourusername at hotel.futuregrid.org:nimbus_creds.tar.gz .
-$ tar xvfz nimbus_creds.tar.gz
------
-When you extract your credential file, look at the file called
-hotel.conf. Near the bottom of this file will be two settings
-called vws.repository.s3id and vws.repository.s3key. Copy these
-values for the next step.
-
-Configuring Swift VM Tools
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-Change directories into the swift-vm-boot directory. Edit the file
-called env.sh. There are two settings here that you will need to
-modify: FUTUREGRID_IAAS_ACCESS_KEY and FUTUREGRID_IAAS_SECRET_KEY.
-Paste your access key (s3id) and secret key (s3key) here and save
-the file. By default, env.sh requests 2 nodes on hotel and 2 nodes
-on sierra. Change these values as needed. Run install.sh from the
-swift-vm-boot directory to complete the installation of Swift VM
-Tools. The Swift VM Tools may require a fairly recent version of
-Python. If you run into problems while running the install.sh
-script, please try a more recent version of Python and associated
-libraries.
-
-Configuring coaster-service.conf
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Within your swift distribution's etc directory is a file called
-coaster-service.conf. There are many options here you can modify
-as needed, but these are the settings that will be required
-to run on futuregrid:
-
------
-export WORKER_WORK=/tmp
-export WORKER_MODE=futuregrid
-export SHARED_FILESYSTEM=no
-export WORKER_USERNAME=root
-export SSH_TUNNELING=yes
-export SWIFTVMBOOT_DIR=$HOME/swift-vm-boot
-export WORK=/tmp
------
-
-These is a brief description of these settings in the coaster-service.conf file.
-Tunneling should be used when you are restricted by a firewall. If your local
-machine has multiple network interfaces, you should also manually set the IP
-address of your machine with export IPADDR=your.ip.address.here.
-
-Starting the Coaster Service Script
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Now that everything is configured, change to a temporary directory
-and run this command to start the coaster service:
-
------
-$ start-coaster-service
------
-
-This command will start the VMs, start the required processes on the worker nodes,
-and generate Swift configuration files for you to use. The configuration files
-will be generated in your current directory. These files are sites.xml, tc.data,
-and cf.
-
-Running Swift
-~~~~~~~~~~~~~
-Now that you have all of your configuration files generated, run the following command:
-
------
-$ swift -sites.file sites.xml -tc.file tc.data -config cf <yourscript.swift>
------
-
-If you would like to create a custom tc file for repeated use, rename it to something other
-than tc.data to prevent it from being overwritten. The sites.xml however will need to be
-regenerated every time you start the coaster service. If you need to repeatedly modify some
-sites.xml options, you may edit the template in Swift's etc/sites/persistent-coasters. You
-may also create your own custom tc files with the hostname of persistent-coasters. More
-information about this can be found in the Swift userguide at
-http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html.
-
-Stopping the Coaster Service Script
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-To stop the coaster service, run the following command:
------
-$ stop-coaster-service
------
-
-This will kill the coaster service, kill the worker scripts on remote systems and terminate
-the virtual machines that were created during start-coaster-service.
-
-More Help
-~~~~~~~~~
-The best place for additional help is the Swift user mailing list. You can subscribe to this list at
-http://mail.ci.uchicago.edu/mailman/listinfo/swift-user. When submitting information, please send
-your sites.xml file, your tc.data, and any error messages you run into.
-
Copied: trunk/docs/siteguide/futuregrid (from rev 5078, branches/release-0.93/docs/siteguide/futuregrid)
===================================================================
--- trunk/docs/siteguide/futuregrid (rev 0)
+++ trunk/docs/siteguide/futuregrid 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,134 @@
+Futuregrid Quickstart Guide
+---------------------------
+
+Downloading and Building Swift
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The most recent versions of Swift can be found at
+http://www.ci.uchicago.edu/swift/downloads/index.php. To run on futuregrid,
+you will need to download the development/trunk version from SVN.
+
+Adding Swift to your PATH
+~~~~~~~~~~~~~~~~~~~~~~~~~
+Once you have installed Swift, add the Swift binary to your PATH so you can
+easily run it from any directory.
+
+In your home directory, edit the file ".bashrc".
+
+If you have installed Swift via a source repository, add the following line
+at the bottom of .bashrc.
+
+-----
+export PATH=$PATH:$HOME/cog/modules/swift/dist/swift-svn/bin
+-----
+
+Requesting Futuregrid Access
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+If you do not already have a futuregrid account, you can follow the
+instructions at https://portal.futuregrid.org/gettingstarted to get started.
+This page provides information on how to create an account, how to join
+a project, how to set up your SSH keys, and how to create a new project.
+
+Downloading Swift VM Tools
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+A set of scripts based around cloudinitd are used to easily start virtual
+machines. To download, change to your home directory and run the
+following command:
+
+-----
+$ svn co https://svn.ci.uchicago.edu/svn/vdl2/usertools/swift-vm-boot
+-----
+
+Download your Credentials
+~~~~~~~~~~~~~~~~~~~~~~~~~
+Run the following commands to retrieve your credentials:
+-----
+$ scp yourusername at hotel.futuregrid.org:nimbus_creds.tar.gz .
+$ tar xvfz nimbus_creds.tar.gz
+-----
+When you extract your credential file, look at the file called
+hotel.conf. Near the bottom of this file will be two settings
+called vws.repository.s3id and vws.repository.s3key. Copy these
+values for the next step.
+
+Configuring Swift VM Tools
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+Change directories into the swift-vm-boot directory. Edit the file
+called env.sh. There are two settings here that you will need to
+modify: FUTUREGRID_IAAS_ACCESS_KEY and FUTUREGRID_IAAS_SECRET_KEY.
+Paste your access key (s3id) and secret key (s3key) here and save
+the file. By default, env.sh requests 2 nodes on hotel and 2 nodes
+on sierra. Change these values as needed. Run install.sh from the
+swift-vm-boot directory to complete the installation of Swift VM
+Tools. The Swift VM Tools may require a fairly recent version of
+Python. If you run into problems while running the install.sh
+script, please try a more recent version of Python and associated
+libraries.
+
+Configuring coaster-service.conf
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Within your swift distribution's etc directory is a file called
+coaster-service.conf. There are many options here you can modify
+as needed, but these are the settings that will be required
+to run on futuregrid:
+
+-----
+export WORKER_WORK=/tmp
+export WORKER_MODE=futuregrid
+export SHARED_FILESYSTEM=no
+export WORKER_USERNAME=root
+export SSH_TUNNELING=yes
+export SWIFTVMBOOT_DIR=$HOME/swift-vm-boot
+export WORK=/tmp
+-----
+
+These is a brief description of these settings in the coaster-service.conf file.
+Tunneling should be used when you are restricted by a firewall. If your local
+machine has multiple network interfaces, you should also manually set the IP
+address of your machine with export IPADDR=your.ip.address.here.
+
+Starting the Coaster Service Script
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Now that everything is configured, change to a temporary directory
+and run this command to start the coaster service:
+
+-----
+$ start-coaster-service
+-----
+
+This command will start the VMs, start the required processes on the worker nodes,
+and generate Swift configuration files for you to use. The configuration files
+will be generated in your current directory. These files are sites.xml, tc.data,
+and cf.
+
+Running Swift
+~~~~~~~~~~~~~
+Now that you have all of your configuration files generated, run the following command:
+
+-----
+$ swift -sites.file sites.xml -tc.file tc.data -config cf <yourscript.swift>
+-----
+
+If you would like to create a custom tc file for repeated use, rename it to something other
+than tc.data to prevent it from being overwritten. The sites.xml however will need to be
+regenerated every time you start the coaster service. If you need to repeatedly modify some
+sites.xml options, you may edit the template in Swift's etc/sites/persistent-coasters. You
+may also create your own custom tc files with the hostname of persistent-coasters. More
+information about this can be found in the Swift userguide at
+http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html.
+
+Stopping the Coaster Service Script
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+To stop the coaster service, run the following command:
+-----
+$ stop-coaster-service
+-----
+
+This will kill the coaster service, kill the worker scripts on remote systems and terminate
+the virtual machines that were created during start-coaster-service.
+
+More Help
+~~~~~~~~~
+The best place for additional help is the Swift user mailing list. You can subscribe to this list at
+http://mail.ci.uchicago.edu/mailman/listinfo/swift-user. When submitting information, please send
+your sites.xml file, your tc.data, and any error messages you run into.
+
Deleted: trunk/docs/siteguide/images
===================================================================
--- branches/release-0.93/docs/siteguide/images 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/siteguide/images 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1 +0,0 @@
-link ../tutorial/images
\ No newline at end of file
Copied: trunk/docs/siteguide/images (from rev 5078, branches/release-0.93/docs/siteguide/images)
===================================================================
--- trunk/docs/siteguide/images (rev 0)
+++ trunk/docs/siteguide/images 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1 @@
+link ../tutorial/images
\ No newline at end of file
Deleted: trunk/docs/siteguide/intrepid
===================================================================
--- branches/release-0.93/docs/siteguide/intrepid 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/siteguide/intrepid 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,168 +0,0 @@
-Intrepid
---------
-
-Requesting Access
-~~~~~~~~~~~~~~~~~
-If you do not already have an account on Intrepid, you can request
-one at https://accounts.alcf.anl.gov/accounts/request.php. More information about
-this process and requesting allocations for your project can be found at
-http://www.alcf.anl.gov/support/gettingstarted/index.php.
-
-SSH Keys
-~~~~~~~~
-Accessing the Intrepid via SSH can be done with any SSH software package.
-Before logging in, you will need to generate an SSH public key and send it to
-support at alcf.anl.gov for verification and installation.
-
-Cryptocard
-~~~~~~~~~~
-This security token uses one-time passwords for controlled access to the BG/P login systems.
-
-Connecting to a login node
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-When you gain access to Intrepid, you should receive a cryptocard and a temporary PIN.
-You must have a working cryptocard, know your PIN, and have your SSH key in place before
-you may login.
-
-You can connect to Intrepid with the following command:
-
------
-ssh yourusername at intrepid.alcf.anl.gov
------
-
-You will be presented with a password prompt. The first part of your password is your PIN. Enter you PIN,
-press the Cryptocard button, and then enter the password your crypocard generates. If this is the first
-time you are logging in, you will be prompted to change your PIN.
-
-Downloading and building Swift
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The most recent versions of Swift can be found at
-http://www.ci.uchicago.edu/swift/downloads/index.php. Follow the instructions
-provided on that site to download and build Swift.
-
-Adding Swift to your PATH
-~~~~~~~~~~~~~~~~~~~~~~~~~
-Once you have installed Swift, add the Swift binary to your PATH so you can
-easily run it from any directory.
-
-In your home directory, edit the file ".bashrc".
-
-If you have installed Swift via a source repository, add the following line
-at the bottom of .bashrc.
-
------
-export PATH=$PATH:$HOME/cog/modules/swift/dist/swift-svn/bin
------
-
-If you have installed Swift via a binary package, add this line:
-
------
-export PATH=$PATH:$HOME/swift-<version>/bin
------
-
-Replace <version> with the actual name of the swift directory in the example above.
-
-What You Need To Know Before Running Swift
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Before you can create a Swift configuration file, there are some things
-you will need to know.
-
-Swift Work Directory
-^^^^^^^^^^^^^^^^^^^^
-The Swift work directory is a directory which Swift uses for processing work.
-This directory needs to be writable. Common options for this are:
-
------
-/home/username/swiftwork
-/home/username/work
-/tmp
------
-
-Which project(s) are you a member of?
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Intrepid requires that you are a member of a project. You can determine this by
-running the following command:
-
------
-$ projects
-HTCScienceApps
------
-
-If you are not a member of a project, you must first request access
-to a project. More information on this process can be found at
-https://wiki.alcf.anl.gov/index.php/Discretionary_Allocations
-
-Determine your Queue
-^^^^^^^^^^^^^^^^^^^^
-
-Intrepid has several different queues you can submit jobs to depending on
-the type of work you will be doing. The command "qstat -q" will print
-the most up to date list of this information.
-
-.Intrepid Queues
-[options="header"]
-|=========================================================
-|User Queue|Queue|Nodes|Time (hours)|User Maxrun|Project maxrun
-|prod-devel|prod-devel|64-512|0-1|5|20
-|prod|prod-short|512-4096|0-6|5|20
-|prod|prod-long|512-4096|6-12|5|20
-|prod|prod-capability|4097-32768|0-12|2|20
-|prod|prod-24k|16385-24576|0-12|2|20
-|prod|prod-bigrun|32769-40960|0-12|2|20
-|prod|backfill|512-8192|0-6|5|10
-|============================================================
-
-Generating Configuration Files
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Now that you know what queue to use, your project, and your work directory, it is time to
-set up Swift. Swift uses a configuration file called sites.xml to determine how it should run.
-There are two methods you can use for creating this file. You can manually edit
-the configuration file, or generate it with a utility called gensites.
-
-Manually Editing sites.xml
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-Below is the template that is used by Swift's test suite for running on Intrepid.
-TODO: Update the rest below here
-
------
-include::../../tests/providers/intrepid/sites.template.xml[]
------
-
-The values to note here are the ones that are listed between underscores. In the example above, they are \_QUEUE_, and \_WORK_. Queue is the PADS queue to use and WORK is the swift work directory. These are placeholder values you will need to modify to fit your needs. Copy and paste this template, replace the values, and call it sites.xml.
-
-Manually Editing tc.data
-~~~~~~~~~~~~~~~~~~~~~~~~
-Below is the tc.data file used by Swift's test suite for running on PADS.
-
------
-include::../../tests/providers/intrepid/tc.template.data[]
-------
-
-Copy these commands and save it as tc.data.
-
-Catsn.swift
-~~~~~~~~~~~
-The swift script we will run is called catsn.swift. It simply cats a file and saves the result. This is a nice simple test to ensure jobs are running correctly. Create a file called data.txt which contains some simple input - a "hello world" will do the trick.
-
------
-include::../../examples/misc/catsn.swift[]
------
-
-Running Swift
-~~~~~~~~~~~~~
-Now that everything is in place, run Swift with the following command:
-
------
-swift -sites.file sites.xml -tc.file tc.data catsn.swift -n=10
------
-
-You should see several new files being created, called catsn.0001.out, catsn.0002.out, etc. Each of these
-files should contain the contents of what you placed into data.txt. If this happens, your job has run
-successfully on PADS!
-
-More Help
-~~~~~~~~~
-The best place for additional help is the Swift user mailing list. You can subscribe to this list at
-https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user. When submitting information, please send your sites.xml file, your tc.data, and any Swift log files that were created during your attempt.
-
-
Copied: trunk/docs/siteguide/intrepid (from rev 5078, branches/release-0.93/docs/siteguide/intrepid)
===================================================================
--- trunk/docs/siteguide/intrepid (rev 0)
+++ trunk/docs/siteguide/intrepid 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,168 @@
+Intrepid
+--------
+
+Requesting Access
+~~~~~~~~~~~~~~~~~
+If you do not already have an account on Intrepid, you can request
+one at https://accounts.alcf.anl.gov/accounts/request.php. More information about
+this process and requesting allocations for your project can be found at
+http://www.alcf.anl.gov/support/gettingstarted/index.php.
+
+SSH Keys
+~~~~~~~~
+Accessing the Intrepid via SSH can be done with any SSH software package.
+Before logging in, you will need to generate an SSH public key and send it to
+support at alcf.anl.gov for verification and installation.
+
+Cryptocard
+~~~~~~~~~~
+This security token uses one-time passwords for controlled access to the BG/P login systems.
+
+Connecting to a login node
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+When you gain access to Intrepid, you should receive a cryptocard and a temporary PIN.
+You must have a working cryptocard, know your PIN, and have your SSH key in place before
+you may login.
+
+You can connect to Intrepid with the following command:
+
+-----
+ssh yourusername at intrepid.alcf.anl.gov
+-----
+
+You will be presented with a password prompt. The first part of your password is your PIN. Enter you PIN,
+press the Cryptocard button, and then enter the password your crypocard generates. If this is the first
+time you are logging in, you will be prompted to change your PIN.
+
+Downloading and building Swift
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The most recent versions of Swift can be found at
+http://www.ci.uchicago.edu/swift/downloads/index.php. Follow the instructions
+provided on that site to download and build Swift.
+
+Adding Swift to your PATH
+~~~~~~~~~~~~~~~~~~~~~~~~~
+Once you have installed Swift, add the Swift binary to your PATH so you can
+easily run it from any directory.
+
+In your home directory, edit the file ".bashrc".
+
+If you have installed Swift via a source repository, add the following line
+at the bottom of .bashrc.
+
+-----
+export PATH=$PATH:$HOME/cog/modules/swift/dist/swift-svn/bin
+-----
+
+If you have installed Swift via a binary package, add this line:
+
+-----
+export PATH=$PATH:$HOME/swift-<version>/bin
+-----
+
+Replace <version> with the actual name of the swift directory in the example above.
+
+What You Need To Know Before Running Swift
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Before you can create a Swift configuration file, there are some things
+you will need to know.
+
+Swift Work Directory
+^^^^^^^^^^^^^^^^^^^^
+The Swift work directory is a directory which Swift uses for processing work.
+This directory needs to be writable. Common options for this are:
+
+-----
+/home/username/swiftwork
+/home/username/work
+/tmp
+-----
+
+Which project(s) are you a member of?
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Intrepid requires that you are a member of a project. You can determine this by
+running the following command:
+
+-----
+$ projects
+HTCScienceApps
+-----
+
+If you are not a member of a project, you must first request access
+to a project. More information on this process can be found at
+https://wiki.alcf.anl.gov/index.php/Discretionary_Allocations
+
+Determine your Queue
+^^^^^^^^^^^^^^^^^^^^
+
+Intrepid has several different queues you can submit jobs to depending on
+the type of work you will be doing. The command "qstat -q" will print
+the most up to date list of this information.
+
+.Intrepid Queues
+[options="header"]
+|=========================================================
+|User Queue|Queue|Nodes|Time (hours)|User Maxrun|Project maxrun
+|prod-devel|prod-devel|64-512|0-1|5|20
+|prod|prod-short|512-4096|0-6|5|20
+|prod|prod-long|512-4096|6-12|5|20
+|prod|prod-capability|4097-32768|0-12|2|20
+|prod|prod-24k|16385-24576|0-12|2|20
+|prod|prod-bigrun|32769-40960|0-12|2|20
+|prod|backfill|512-8192|0-6|5|10
+|============================================================
+
+Generating Configuration Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Now that you know what queue to use, your project, and your work directory, it is time to
+set up Swift. Swift uses a configuration file called sites.xml to determine how it should run.
+There are two methods you can use for creating this file. You can manually edit
+the configuration file, or generate it with a utility called gensites.
+
+Manually Editing sites.xml
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+Below is the template that is used by Swift's test suite for running on Intrepid.
+TODO: Update the rest below here
+
+-----
+include::../../tests/providers/intrepid/sites.template.xml[]
+-----
+
+The values to note here are the ones that are listed between underscores. In the example above, they are \_QUEUE_, and \_WORK_. Queue is the PADS queue to use and WORK is the swift work directory. These are placeholder values you will need to modify to fit your needs. Copy and paste this template, replace the values, and call it sites.xml.
+
+Manually Editing tc.data
+~~~~~~~~~~~~~~~~~~~~~~~~
+Below is the tc.data file used by Swift's test suite for running on PADS.
+
+-----
+include::../../tests/providers/intrepid/tc.template.data[]
+------
+
+Copy these commands and save it as tc.data.
+
+Catsn.swift
+~~~~~~~~~~~
+The swift script we will run is called catsn.swift. It simply cats a file and saves the result. This is a nice simple test to ensure jobs are running correctly. Create a file called data.txt which contains some simple input - a "hello world" will do the trick.
+
+-----
+include::../../examples/misc/catsn.swift[]
+-----
+
+Running Swift
+~~~~~~~~~~~~~
+Now that everything is in place, run Swift with the following command:
+
+-----
+swift -sites.file sites.xml -tc.file tc.data catsn.swift -n=10
+-----
+
+You should see several new files being created, called catsn.0001.out, catsn.0002.out, etc. Each of these
+files should contain the contents of what you placed into data.txt. If this happens, your job has run
+successfully on PADS!
+
+More Help
+~~~~~~~~~
+The best place for additional help is the Swift user mailing list. You can subscribe to this list at
+https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user. When submitting information, please send your sites.xml file, your tc.data, and any Swift log files that were created during your attempt.
+
+
Deleted: trunk/docs/siteguide/pads
===================================================================
--- branches/release-0.93/docs/siteguide/pads 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/siteguide/pads 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,246 +0,0 @@
-PADS
-----
-
-PADS Quickstart
-~~~~~~~~~~~~~~~
-This section will walk you through running a simple Swift script
-on PADS. It will explain how to setup software packages, how
-to create the required Swift configuration files, and finally
-how to run Swift.
-
-Adding Software Packages
-^^^^^^^^^^^^^^^^^^^^^^^^
-In your home directory, edit a file called .soft and add these
-lines (in this order):
------
-+java-sun
-+maui
-+torque
- at default
------
-
-Log out of PADS, and log back in for these changes to take effect.
-
-Creating sites.xml
-^^^^^^^^^^^^^^^^^^
-Swift relies on various configuration files to determine how to
-run. This section will provide a working configuration file which
-you can copy and paste to get running quickly. The sites.xml file
-tells Swift how to submit jobs, where working directories are
-located, and various other configuration information. More
-information on sites.xml can be found in the Swift User's Guide.
-
-The first step is to paste the text below into a file named sites.xml.
-
------
-include::../../tests/providers/PADS/coasters/sites.template.xml[]
------
-
-This file will require just a few customizations. First, create a
-directory called swiftwork. Modify \_WORK_ in sites.xml
-to point to this new directory. For example
------
-<workdirectory>/home/myhome/swiftwork</workdirectory>
------
-
-Creating tc.data
-^^^^^^^^^^^^^^^^
-The tc.data configuration file gives information about the applications
-that will be called by Swift. More information about the format
-of tc.data can be found in the Swift User's guide.
-
-Paste the following example into a file named tc.data
-
------
-include::../../tests/providers/PADS/coasters/tc.template.data[]
------
-
-Copy a Swift Script
-^^^^^^^^^^^^^^^^^^^
-
-Within the Swift directory is an examples directory which contains
-several introductory Swift scripts. The example we will use in this
-section is called catsn.swift. Copy this script to the same directory
-that your sites.xml and tc.data files are located.
-
------
-$ cp ~/swift-0.93/examples/misc/catsn.swift .
-$ cp ~/swift-0.93/examples/misc/data.txt .
------
-TIP: The location of your swift directory may vary depending on how you installed it. Change this to the examples/misc directory of your installation as needed.
-
-Run Swift
-^^^^^^^^^
-
-Finally, run the script
------
-$ swift -sites.file sites.xml -tc.file tc.data catsn.swift
------
-
-You should see 10 new text files get created, named catsn*.out. If
-you see these files, then you have succesfully run Swift on PADS!
-
-TIP: Make sure your default project is defined. Read on for more information.
-
-Read on for more detailed information about running Swift on PADS.
-
-Requesting Access
-~~~~~~~~~~~~~~~~~
-If you do not already have a Computation Institute account, you can request
-one at https://www.ci.uchicago.edu/accounts. This page will give you a list
-of resources you can request access to. Be sure that PADS is selected. If
-you already have an existing CI account, but do not have access to PADS,
-send an email to support at ci.uchicago.edu to request access.
-
-SSH Keys
-~~~~~~~~
-Before accessing PADS, be sure to have your SSH keys configured correctly.
-There is some basic information about SSH and how to generate your key at
-http://www.ci.uchicago.edu/wiki/bin/view/Resources/SshKeys. Once you have
-followed those instructions, you can add your key at
-https://www.ci.uchicago.edu/support/sshkeys/.
-
-Connecting to a login node
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-Once your keys are configured, you should be able to access a PADS login
-node with the following command:
-
------
-ssh yourusername at login.pads.ci.uchicago.edu
------
-
-Configuring softenv
-~~~~~~~~~~~~~~~~~~~
-Softenv is a system used for managing applications. In order to run Swift,
-the softenv environment will have to be modified slightly. Softenv is
-configured by a file in your home directory called .soft. Edit this file
-to look like this:
------
-+java-sun
-+maui
-+torque
- at default
------
-
-Swift Work Directory
-^^^^^^^^^^^^^^^^^^^^
-The Swift work directory is a directory which Swift uses for processing work.
-This directory needs to be writable and available to all worker nodes on
-a shared filesystem.
-
-.Examples
------
-/home/username/swiftwork
-/home/username/work
-/autonfs/gpfs-pads/projects/CI-CCR000013/myusername
------
-
-Which project(s) are you a member of?
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-PADS requires that you are a member of a project. You can determine this by
-running the following command:
-
------
-$ projects --available
-
-The following projects are available for your use
-
-Project PI Title
-
-CI-CCR000013 Michael Wilde The Swift Parallel Scripting System
------
-
-If you are not a member of a project, you must first request access
-to a project at http://www.ci.uchicago.edu/hpc/projects.
-
-You should make sure that you have a project set as default. Run
-the projects command with no arguments to determine if you have a default.
-
-------
-$ projects
-You have no default project set.
------
-
-To set your default project, use projects --set
-------
-$ projects --set CI-CCR000013 --all
-Your default project for all CI clusters has been set to CI-CCR000013.
------
-
-Determine your Queue
-^^^^^^^^^^^^^^^^^^^^
-
-PADS has several different queues you can submit jobs to depending on
-the type of work you will be doing. The command "qstat -q" will print
-the most up to date list of this information.
-
-.PADS Queues
-[options="header"]
-|=========================================================
-|Queue |Memory|CPU Time|Walltime|Node|Run|Que|Lm |State
-|route |-- |-- |-- |-- | 0| 0|-- | E R
-|short |-- |-- |04:00:00|-- | 64| 0|-- | E R
-|extended|-- |-- |-- |-- | 0| 0|-- | E R
-|fast |-- |-- |01:00:00|1 | 0|152|-- | E R
-|long |-- |-- |24:00:00|-- |232|130|-- | E R
-|=========================================================
-
-Generating Configuration Files
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Now that you know what queue to use, your project, and your work directory, it is time to
-set up Swift. Swift uses a configuration file called sites.xml to determine how it should run.
-There are two methods you can use for creating this file. You can manually edit
-the configuration file, or generate it with a utility called gensites.
-
-Manually Editing sites.xml
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-Below is the template that is used by Swift's test suite for running on PADS.
-
------
-include::../../tests/providers/PADS/coasters/sites.template.xml[]
------
-
-Before you can use this configuration file, you will need to modify \_WORK_ to a directory writable by you,
-and have your default project defined. Copy this template, replace \_WORK_, and then save as sites.xml.
-
-If you wanted to use a different queue, replace "fast" with the desired queue name.
-
-Manually Editing tc.data
-~~~~~~~~~~~~~~~~~~~~~~~~
-Below is the tc.data file used by Swift's test suite for running on PADS.
-
------
-include::../../tests/providers/PADS/coasters/tc.template.data[]
-------
-
-Copy these commands and save it as tc.data.
-
-Catsn.swift
-~~~~~~~~~~~
-The swift script we will run is called catsn.swift. It simply cats a file and saves the result. This is
-a nice simple test to ensure jobs are running correctly. Create a file called data.txt which contains
-some simple input - a "hello world" will do the trick.
-
------
-include::../../examples/misc/catsn.swift[]
------
-
-
-Running Swift
-~~~~~~~~~~~~~
-Now that everything is in place, run Swift with the following command:
-
------
-swift -sites.file sites.xml -tc.file tc.data catsn.swift -n=10
------
-
-You should see several new files being created, called catsn.0001.out, catsn.0002.out, etc. Each of these
-files should contain the contents of what you placed into data.txt. If this happens, your job has run
-successfully on PADS!
-
-More Help
-~~~~~~~~~
-The best place for additional help is the Swift user mailing list. You can subscribe to this list at
-https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user. When submitting information, please send your sites.xml file, your tc.data, and any Swift log files that were created during your attempt.
-
-
Copied: trunk/docs/siteguide/pads (from rev 5078, branches/release-0.93/docs/siteguide/pads)
===================================================================
--- trunk/docs/siteguide/pads (rev 0)
+++ trunk/docs/siteguide/pads 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,246 @@
+PADS
+----
+
+PADS Quickstart
+~~~~~~~~~~~~~~~
+This section will walk you through running a simple Swift script
+on PADS. It will explain how to setup software packages, how
+to create the required Swift configuration files, and finally
+how to run Swift.
+
+Adding Software Packages
+^^^^^^^^^^^^^^^^^^^^^^^^
+In your home directory, edit a file called .soft and add these
+lines (in this order):
+-----
++java-sun
++maui
++torque
+ at default
+-----
+
+Log out of PADS, and log back in for these changes to take effect.
+
+Creating sites.xml
+^^^^^^^^^^^^^^^^^^
+Swift relies on various configuration files to determine how to
+run. This section will provide a working configuration file which
+you can copy and paste to get running quickly. The sites.xml file
+tells Swift how to submit jobs, where working directories are
+located, and various other configuration information. More
+information on sites.xml can be found in the Swift User's Guide.
+
+The first step is to paste the text below into a file named sites.xml.
+
+-----
+include::../../tests/providers/PADS/coasters/sites.template.xml[]
+-----
+
+This file will require just a few customizations. First, create a
+directory called swiftwork. Modify \_WORK_ in sites.xml
+to point to this new directory. For example
+-----
+<workdirectory>/home/myhome/swiftwork</workdirectory>
+-----
+
+Creating tc.data
+^^^^^^^^^^^^^^^^
+The tc.data configuration file gives information about the applications
+that will be called by Swift. More information about the format
+of tc.data can be found in the Swift User's guide.
+
+Paste the following example into a file named tc.data
+
+-----
+include::../../tests/providers/PADS/coasters/tc.template.data[]
+-----
+
+Copy a Swift Script
+^^^^^^^^^^^^^^^^^^^
+
+Within the Swift directory is an examples directory which contains
+several introductory Swift scripts. The example we will use in this
+section is called catsn.swift. Copy this script to the same directory
+that your sites.xml and tc.data files are located.
+
+-----
+$ cp ~/swift-0.93/examples/misc/catsn.swift .
+$ cp ~/swift-0.93/examples/misc/data.txt .
+-----
+TIP: The location of your swift directory may vary depending on how you installed it. Change this to the examples/misc directory of your installation as needed.
+
+Run Swift
+^^^^^^^^^
+
+Finally, run the script
+-----
+$ swift -sites.file sites.xml -tc.file tc.data catsn.swift
+-----
+
+You should see 10 new text files get created, named catsn*.out. If
+you see these files, then you have succesfully run Swift on PADS!
+
+TIP: Make sure your default project is defined. Read on for more information.
+
+Read on for more detailed information about running Swift on PADS.
+
+Requesting Access
+~~~~~~~~~~~~~~~~~
+If you do not already have a Computation Institute account, you can request
+one at https://www.ci.uchicago.edu/accounts. This page will give you a list
+of resources you can request access to. Be sure that PADS is selected. If
+you already have an existing CI account, but do not have access to PADS,
+send an email to support at ci.uchicago.edu to request access.
+
+SSH Keys
+~~~~~~~~
+Before accessing PADS, be sure to have your SSH keys configured correctly.
+There is some basic information about SSH and how to generate your key at
+http://www.ci.uchicago.edu/wiki/bin/view/Resources/SshKeys. Once you have
+followed those instructions, you can add your key at
+https://www.ci.uchicago.edu/support/sshkeys/.
+
+Connecting to a login node
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+Once your keys are configured, you should be able to access a PADS login
+node with the following command:
+
+-----
+ssh yourusername at login.pads.ci.uchicago.edu
+-----
+
+Configuring softenv
+~~~~~~~~~~~~~~~~~~~
+Softenv is a system used for managing applications. In order to run Swift,
+the softenv environment will have to be modified slightly. Softenv is
+configured by a file in your home directory called .soft. Edit this file
+to look like this:
+-----
++java-sun
++maui
++torque
+ at default
+-----
+
+Swift Work Directory
+^^^^^^^^^^^^^^^^^^^^
+The Swift work directory is a directory which Swift uses for processing work.
+This directory needs to be writable and available to all worker nodes on
+a shared filesystem.
+
+.Examples
+-----
+/home/username/swiftwork
+/home/username/work
+/autonfs/gpfs-pads/projects/CI-CCR000013/myusername
+-----
+
+Which project(s) are you a member of?
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+PADS requires that you are a member of a project. You can determine this by
+running the following command:
+
+-----
+$ projects --available
+
+The following projects are available for your use
+
+Project PI Title
+
+CI-CCR000013 Michael Wilde The Swift Parallel Scripting System
+-----
+
+If you are not a member of a project, you must first request access
+to a project at http://www.ci.uchicago.edu/hpc/projects.
+
+You should make sure that you have a project set as default. Run
+the projects command with no arguments to determine if you have a default.
+
+------
+$ projects
+You have no default project set.
+-----
+
+To set your default project, use projects --set
+------
+$ projects --set CI-CCR000013 --all
+Your default project for all CI clusters has been set to CI-CCR000013.
+-----
+
+Determine your Queue
+^^^^^^^^^^^^^^^^^^^^
+
+PADS has several different queues you can submit jobs to depending on
+the type of work you will be doing. The command "qstat -q" will print
+the most up to date list of this information.
+
+.PADS Queues
+[options="header"]
+|=========================================================
+|Queue |Memory|CPU Time|Walltime|Node|Run|Que|Lm |State
+|route |-- |-- |-- |-- | 0| 0|-- | E R
+|short |-- |-- |04:00:00|-- | 64| 0|-- | E R
+|extended|-- |-- |-- |-- | 0| 0|-- | E R
+|fast |-- |-- |01:00:00|1 | 0|152|-- | E R
+|long |-- |-- |24:00:00|-- |232|130|-- | E R
+|=========================================================
+
+Generating Configuration Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Now that you know what queue to use, your project, and your work directory, it is time to
+set up Swift. Swift uses a configuration file called sites.xml to determine how it should run.
+There are two methods you can use for creating this file. You can manually edit
+the configuration file, or generate it with a utility called gensites.
+
+Manually Editing sites.xml
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+Below is the template that is used by Swift's test suite for running on PADS.
+
+-----
+include::../../tests/providers/PADS/coasters/sites.template.xml[]
+-----
+
+Before you can use this configuration file, you will need to modify \_WORK_ to a directory writable by you,
+and have your default project defined. Copy this template, replace \_WORK_, and then save as sites.xml.
+
+If you wanted to use a different queue, replace "fast" with the desired queue name.
+
+Manually Editing tc.data
+~~~~~~~~~~~~~~~~~~~~~~~~
+Below is the tc.data file used by Swift's test suite for running on PADS.
+
+-----
+include::../../tests/providers/PADS/coasters/tc.template.data[]
+------
+
+Copy these commands and save it as tc.data.
+
+Catsn.swift
+~~~~~~~~~~~
+The swift script we will run is called catsn.swift. It simply cats a file and saves the result. This is
+a nice simple test to ensure jobs are running correctly. Create a file called data.txt which contains
+some simple input - a "hello world" will do the trick.
+
+-----
+include::../../examples/misc/catsn.swift[]
+-----
+
+
+Running Swift
+~~~~~~~~~~~~~
+Now that everything is in place, run Swift with the following command:
+
+-----
+swift -sites.file sites.xml -tc.file tc.data catsn.swift -n=10
+-----
+
+You should see several new files being created, called catsn.0001.out, catsn.0002.out, etc. Each of these
+files should contain the contents of what you placed into data.txt. If this happens, your job has run
+successfully on PADS!
+
+More Help
+~~~~~~~~~
+The best place for additional help is the Swift user mailing list. You can subscribe to this list at
+https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user. When submitting information, please send your sites.xml file, your tc.data, and any Swift log files that were created during your attempt.
+
+
Deleted: trunk/docs/siteguide/prereqs
===================================================================
--- branches/release-0.93/docs/siteguide/prereqs 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/siteguide/prereqs 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,8 +0,0 @@
-Prerequisites
--------------
-
-This guide assumes that you have already downloaded and installed Swift.
-It assumes that Swift is in your PATH and that you have a working
-version of Sun Java 1.5+. For more information on downloading and
-installing Swift, please see the
-http://www.ci.uchicago.edu/swift/guides/release-0.93/quickstart/quickstart.html[Swift Quickstart Guide].
Copied: trunk/docs/siteguide/prereqs (from rev 5078, branches/release-0.93/docs/siteguide/prereqs)
===================================================================
--- trunk/docs/siteguide/prereqs (rev 0)
+++ trunk/docs/siteguide/prereqs 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,8 @@
+Prerequisites
+-------------
+
+This guide assumes that you have already downloaded and installed Swift.
+It assumes that Swift is in your PATH and that you have a working
+version of Sun Java 1.5+. For more information on downloading and
+installing Swift, please see the
+http://www.ci.uchicago.edu/swift/guides/release-0.93/quickstart/quickstart.html[Swift Quickstart Guide].
Deleted: trunk/docs/siteguide/siteguide.txt
===================================================================
--- branches/release-0.93/docs/siteguide/siteguide.txt 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/siteguide/siteguide.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,17 +0,0 @@
-Site Configuration Guide
-========================
-
-:toc:
-:icons:
-:website: http://www.ci.uchicago.edu/swift/guides/siteguide.php
-:numbered:
-
-include::prereqs[]
-
-include::pads[]
-
-include::beagle[]
-
-include::futuregrid[]
-
-include::intrepid[]
Copied: trunk/docs/siteguide/siteguide.txt (from rev 5078, branches/release-0.93/docs/siteguide/siteguide.txt)
===================================================================
--- trunk/docs/siteguide/siteguide.txt (rev 0)
+++ trunk/docs/siteguide/siteguide.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,17 @@
+Site Configuration Guide
+========================
+
+:toc:
+:icons:
+:website: http://www.ci.uchicago.edu/swift/guides/siteguide.php
+:numbered:
+
+include::prereqs[]
+
+include::pads[]
+
+include::beagle[]
+
+include::futuregrid[]
+
+include::intrepid[]
Deleted: trunk/docs/stylesheets/asciidoc.css
===================================================================
--- branches/release-0.93/docs/stylesheets/asciidoc.css 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/stylesheets/asciidoc.css 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,2 +0,0 @@
-a:link { color:navy; }
-a:visited { color:navy; }
Copied: trunk/docs/stylesheets/asciidoc.css (from rev 5078, branches/release-0.93/docs/stylesheets/asciidoc.css)
===================================================================
--- trunk/docs/stylesheets/asciidoc.css (rev 0)
+++ trunk/docs/stylesheets/asciidoc.css 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,2 @@
+a:link { color:navy; }
+a:visited { color:navy; }
Property changes on: trunk/docs/tutorial
___________________________________________________________________
Deleted: svn:ignore
- tutorial.html
tutorial.pdf
Deleted: trunk/docs/tutorial/abstract
===================================================================
--- trunk/docs/tutorial/abstract 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/tutorial/abstract 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,7 +0,0 @@
-[abstract]
-Abstract
---------
-This is an introductory tutorial on the use of Swift and its programming
-language SwiftScript.
-
-note: merge with intro?
Copied: trunk/docs/tutorial/abstract (from rev 5078, branches/release-0.93/docs/tutorial/abstract)
===================================================================
--- trunk/docs/tutorial/abstract (rev 0)
+++ trunk/docs/tutorial/abstract 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,7 @@
+[abstract]
+Abstract
+--------
+This is an introductory tutorial on the use of Swift and its programming
+language SwiftScript.
+
+note: merge with intro?
Deleted: trunk/docs/tutorial/bits
===================================================================
--- trunk/docs/tutorial/bits 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/tutorial/bits 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,15 +0,0 @@
-Bits
-----
-
-Named and optional parameters
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-In addition to specifying parameters positionally, parameters can be
-named, and if desired a default value can be specified:
-
-.default.swift
-**************
-----
-include::../../examples/tutorial/default.swift[]
-----
-**************
Copied: trunk/docs/tutorial/bits (from rev 5078, branches/release-0.93/docs/tutorial/bits)
===================================================================
--- trunk/docs/tutorial/bits (rev 0)
+++ trunk/docs/tutorial/bits 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,15 @@
+Bits
+----
+
+Named and optional parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In addition to specifying parameters positionally, parameters can be
+named, and if desired a default value can be specified:
+
+.default.swift
+**************
+----
+include::../../examples/tutorial/default.swift[]
+----
+**************
Deleted: trunk/docs/tutorial/hello_world
===================================================================
--- trunk/docs/tutorial/hello_world 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/tutorial/hello_world 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,69 +0,0 @@
-Hello World
------------
-The first example program, hello.swift, outputs a hello world message
-into a file called hello.txt.
-
-.hello.swift
-************
-
-----
-include::../../examples/tutorial/hello.swift[]
-----
-************
-
-To run hello.swift, change directories to the location of the script
-and run the swift command as follows.
-
-TIP: Make sure the bin directory of your swift installation is in your PATH.
-----
-$ cd examples/tutorial
-$ swift hello.swift
-Swift svn swift-r3334 (swift modified locally) cog-r2752
-
-RunID: 20100526-1925-8zjupq1b
-Progress:
-Final status: Finished successfully:1
-$ cat hello.txt
-Hello, world!
-----
-
-The basic structure of this program is a type definition, an
-application procedure definition, a variable definition and then a
-call to the procedure.
-
-First we define a new type, called messagefile. In this example, we will
-use this messagefile type for our output message.
-
-----
-sys::[sed -n '1p' ../../examples/tutorial/hello.swift]
-----
-
-All data in SwiftScript must be typed, whether it is stored in memory or
-on disk. This example defines a very simple type. Later on we will see
-more complex type examples.
-
-----
-sys::[sed -n '3,5p' ../../examples/tutorial/hello.swift]
-----
-
-Next we define a procedure called greeting. This procedure will write
-out the "hello world" message to a file. To achieve this, it executes
-the unix utility 'echo' with a parameter "Hello, world!" and directs
-the standard output into the output file.
-
-The actual file to use is specified by the return parameter, t.
------
-sys::[sed -n '7p' ../../examples/tutorial/hello.swift]
------
-Here we define a variable called outfile. The type of this variable is
-messagefile, and we specify that the contents of this variable will be
-stored on disk in a file called hello.txt
-
------
-sys::[sed -n '3,5p' ../../examples/tutorial/hello.swift]
------
-Now we call the greeting procedure, with its output going to the outfile
-variable and therefore to hello.txt on disk.
-
-Over the following exercises, we'll extend this simple hello world
-program to demonstrate various features of Swift.
Copied: trunk/docs/tutorial/hello_world (from rev 5078, branches/release-0.93/docs/tutorial/hello_world)
===================================================================
--- trunk/docs/tutorial/hello_world (rev 0)
+++ trunk/docs/tutorial/hello_world 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,69 @@
+Hello World
+-----------
+The first example program, hello.swift, outputs a hello world message
+into a file called hello.txt.
+
+.hello.swift
+************
+
+----
+include::../../examples/tutorial/hello.swift[]
+----
+************
+
+To run hello.swift, change directories to the location of the script
+and run the swift command as follows.
+
+TIP: Make sure the bin directory of your swift installation is in your PATH.
+----
+$ cd examples/tutorial
+$ swift hello.swift
+Swift svn swift-r3334 (swift modified locally) cog-r2752
+
+RunID: 20100526-1925-8zjupq1b
+Progress:
+Final status: Finished successfully:1
+$ cat hello.txt
+Hello, world!
+----
+
+The basic structure of this program is a type definition, an
+application procedure definition, a variable definition and then a
+call to the procedure.
+
+First we define a new type, called messagefile. In this example, we will
+use this messagefile type for our output message.
+
+----
+sys::[sed -n '1p' ../../examples/tutorial/hello.swift]
+----
+
+All data in SwiftScript must be typed, whether it is stored in memory or
+on disk. This example defines a very simple type. Later on we will see
+more complex type examples.
+
+----
+sys::[sed -n '3,5p' ../../examples/tutorial/hello.swift]
+----
+
+Next we define a procedure called greeting. This procedure will write
+out the "hello world" message to a file. To achieve this, it executes
+the unix utility 'echo' with a parameter "Hello, world!" and directs
+the standard output into the output file.
+
+The actual file to use is specified by the return parameter, t.
+-----
+sys::[sed -n '7p' ../../examples/tutorial/hello.swift]
+-----
+Here we define a variable called outfile. The type of this variable is
+messagefile, and we specify that the contents of this variable will be
+stored on disk in a file called hello.txt
+
+-----
+sys::[sed -n '3,5p' ../../examples/tutorial/hello.swift]
+-----
+Now we call the greeting procedure, with its output going to the outfile
+variable and therefore to hello.txt on disk.
+
+Over the following exercises, we'll extend this simple hello world
+program to demonstrate various features of Swift.
Deleted: trunk/docs/tutorial/images/icons/caution.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/caution.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/caution.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/images/icons/example.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/example.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/example.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/images/icons/home.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/home.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/home.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/images/icons/important.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/important.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/important.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/images/icons/next.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/next.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/next.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/images/icons/note.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/note.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/note.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/images/icons/prev.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/prev.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/prev.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/images/icons/tip.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/tip.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/tip.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/images/icons/up.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/up.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/up.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/images/icons/warning.png
===================================================================
(Binary files differ)
Copied: trunk/docs/tutorial/images/icons/warning.png (from rev 5078, branches/release-0.93/docs/tutorial/images/icons/warning.png)
===================================================================
(Binary files differ)
Deleted: trunk/docs/tutorial/introduction
===================================================================
--- trunk/docs/tutorial/introduction 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/tutorial/introduction 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,16 +0,0 @@
-Introduction
-------------
-This is an introductory tutorial describing the use of Swift and its programming
-language SwiftScript. It is intended to introduce new users to the basics of Swift.
-It is structured as a series of simple exercises/examples which you can
-try for yourself as you read along.
-
-For information on getting an installation of Swift running, consult the
-http://www.ci.uchicago.edu/swift/guides/quickstartguide.php[Swift Quickstart Guide]. We advise you to install the latest stable release of Swift.
-Return to this document when you have successfully run the test SwiftScript program mentioned there.
-
-There is also a
-sys::[../print_link.sh userguide]
-which contains a more detailed reference material on topics covered in this manual. All
-of the programs included in this tutorial can be found in your Swift
-distribution in the examples/tutorial directory.
Copied: trunk/docs/tutorial/introduction (from rev 5078, branches/release-0.93/docs/tutorial/introduction)
===================================================================
--- trunk/docs/tutorial/introduction (rev 0)
+++ trunk/docs/tutorial/introduction 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,16 @@
+Introduction
+------------
+This is an introductory tutorial describing the use of Swift and its programming
+language SwiftScript. It is intended to introduce new users to the basics of Swift.
+It is structured as a series of simple exercises/examples which you can
+try for yourself as you read along.
+
+For information on getting an installation of Swift running, consult the
+http://www.ci.uchicago.edu/swift/guides/quickstartguide.php[Swift Quickstart Guide]. We advise you to install the latest stable release of Swift.
+Return to this document when you have successfully run the test SwiftScript program mentioned there.
+
+There is also a
+sys::[../print_link.sh userguide]
+which contains a more detailed reference material on topics covered in this manual. All
+of the programs included in this tutorial can be found in your Swift
+distribution in the examples/tutorial directory.
Deleted: trunk/docs/tutorial/language_features
===================================================================
--- trunk/docs/tutorial/language_features 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/tutorial/language_features 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,390 +0,0 @@
-Language features
------------------
-
-Parameters
-~~~~~~~~~~
-
-Procedures can have parameters. Input parameters specify inputs to the
-procedure and output parameters specify outputs. Our hello world greeting
-procedure already uses an output parameter, t, which indicates where the
-greeting output will go. In this section, we will modify the previous
-script to add an input parameter to the greeting function.
-
-.parameter.swift
-****************
-----
-include::../../examples/tutorial/parameter.swift[]
-----
-****************
-
-We have modified the signature of the greeting procedure to indicate
-that it takes a single parameter, s, of type 'string'.
-
-We have modified the invocation of the 'echo' utility so that it takes
-the value of s as a parameter, instead of the string literal "Hello,
-world!".
-
-We have modified the output file definition to point to a different file
-on disk.
-
-We have modified the invocation of greeting so that a greeting string is
-supplied.
-
-The code for this section can be found in parameter.swift. It can be
-invoked using the swift command, with output appearing in parameter.hello.txt:
-
-
-----
-$ swift parameter.swift
-----
-
-Now that we can choose our greeting text, we can call the same procedure
-with different parameters to generate several output files with
-different greetings. The code is in manyparam.swift and can be run as
-before using the swift command.
-
-.manyparam.swift
-****************
-----
-include::../../examples/tutorial/manyparam.swift[]
-----
-****************
-
-Note that we can intermingle definitions of variables with invocations
-of procedures.
-
-When this program runs, there should be three new files in the
-working directory (manyparam.english.txt, manyparam.francais.txt
-and manyparam.nihongo.txt) each containing a greeting in a different
-language.
-
-In addition to specifying parameters positionally, parameters can be
-named, and if desired a default value can be specified.
-
-Adding another application
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-Now we'll define a new application procedure. The procedure we define
-will capitalise all the words in the input file.
-
-To do this, we'll use the unix tr (translate) utility. Here is an
-example of using tr on the unix command line, not using Swift:
-
-----
-$ echo hello | tr '[a-z]' '[A-Z]'
-HELLO
-----
-
-There are two main steps - updating the transformation catalog, and
-updating the application block.
-
-The transformation catalog lists where application executables are located on
-remote sites. We need to modify the transformation catalog to define a logical
-transformation for the tr utility. The transformation catalog can be found
-in etc/tc.data. There are already several entries specifying
-where executables can be found. Add a new line to the file, specifying
-where tr can be found (usually in /usr/bin/tr but it may differ on
-your system), like this:
-
-----
-localhost tr /usr/bin/tr INSTALLED INTEL32::LINUX null
-----
-
-For now, ignore all of the fields except the second and the third. The
-second field tr specifies a logical application name and the third
-specifies the location of the application executable.
-
-Now that we have defined where to find tr, we can use it in SwiftScript.
-
-We can define a new procedure, capitalise, which calls tr.
-
-----
-sys::[sed -n '7,9p' ../../examples/tutorial/capitalise.swift]
-----
-We can call capitalise like this:
-
-----
-sys::[sed -n '12,14p' ../../examples/tutorial/capitalise.swift]
-----
-
-Here is the full program based on this exercise:
-
-.capitalise.swift
-***********************
-----
-include::../../examples/tutorial/capitalise.swift[]
-----
-***********************
-
-Next, run swift and verify the output is correct.
-
-----
-$ swift capitalise.swift
-...
-$ cat capitalise.2.txt
-HELLO FROM SWIFT
-----
-
-Anonymous files
-~~~~~~~~~~~~~~~
-In the previous section, the file hello.txt is used only to store
-an intermediate result. We don't really care about which name is used
-for the file, and we can let Swift choose the name.
-
-To do that, omit the mapping entirely when declaring hellofile:
-
-----
-sys::[sed -n '11p' ../../examples/tutorial/capitalise_anonymous.swift]
-----
-
-Swift will choose a filename, which in the present version will be in a
-subdirectory called _concurrent.
-
-
-Datatypes
-~~~~~~~~~
-All data in variables and files has a data type. So far, we've seen two
-types:
-
- - string - this is a built-in type for storing strings of text in
- memory, much like in other programming languages
- - messagefile - this is a user-defined type used to mark disc resident files as
- containing messages
-
-SwiftScript has the additional built-in types: boolean, integer and
-float that function much like their counterparts in other programming
-languages.
-
-It is also possible to create user defined types with more structure,
-for example:
-
-
-----
-sys::[sed -n '3,6p' ../../examples/tutorial/types.swift]
-----
-
-Each element of the structured type can be accessed using a . like this:
-
-----
-sys::[sed -n '14p' ../../examples/tutorial/types.swift]
-----
-
-The following complete program, types.swift, outputs a greeting using a
-user-defined structure type to hold parameters for the message:
-
-.types.swift
-************
-----
-include::../../examples/tutorial/types.swift[]
-----
-************
-
-Structured types can be comprised of marker types for files. See the
-later section on mappers for more information about this.
-
-
-Arrays
-~~~~~~
-
-We can define arrays using the [] suffix in a variable declaration:
-
-----
-sys::[sed -n '9p' ../../examples/tutorial/arrays.swift]
-----
-
-This program, arrays.swift, will declare an array of message files.
-
-.arrays.swift
-*************
-----
-include::../../examples/tutorial/arrays.swift[]
-----
-*************
-
-Observe that the type of the parameter to greeting is now an array of
-strings, string s[], instead of a single string, string s, that
-elements of the array can be referenced numerically, for example s[0],
-and that the array is initialised using an array literal,
-["how","are","you"].
-
-
-Mappers
-~~~~~~~
-
-A significant difference between SwiftScript and other languages is that
-data can be referred to on disk through variables in a very similar
-fashion to data in memory. For example, in the above examples we have
-seen a variable definition like this:
-
-----
-sys::[sed -n '7p' ../../examples/tutorial/arrays.swift]
-----
-
-This means that outfile is a dataset variable, which is mapped to a
-file on disk called arrays.txt. This variable can be assigned to
-using = in a similar fashion to an in-memory variable. We can say that
-outfile is mapped onto the disk file arrays.txt by a mapper.
-
-There are various ways of mapping in SwiftScript. Two forms of mapping,
-simple named mapping and anonymous mapping, have already
-been seen in this tutorial. Later exercises will introduce more forms.
-
-In simple named mapping, the name of the file that a variable is mapped to
-is explictly listed.
-
-----
-sys::[sed -n '7p' ../../examples/tutorial/hello.swift]
-----
-
-This is useful when you want to explicitly name input and output files
-for your program. An example of this can be seen with 'outfile' in the
-hello world exercise.
-
-With anonymous mapping no name is specified in the source code. A name is
-automatically generated for the file. This is useful for intermediate
-files that are only referenced through SwiftScript.
-A variable declaration is mapped anonymously by ommitting any mapper
-definition.
-
-----
-sys::[sed -n '11p' ../../examples/tutorial/capitalise_anonymous.swift]
-----
-
-Later exercises will introduce other ways of mapping from disk files to
-SwiftScript variables.
-
-
-The Regexp Mapper
-^^^^^^^^^^^^^^^^^
-In this exercise, we introduce the regexp mapper. This mapper
-transforms a string expression using a regular expression, and uses the
-result of that transformation as the filename to map.
-
-regexp.swift demonstrates the use of this by placing output into a
-file that is based on the name of the input file. Our input file is
-mapped to the inputfile variable using the simple named mapper, then
-we use the regular expression mapper to map the output file. We then use
-the countwords() procedure to count the words in the input file and
-store the result in the output file. In order for the countwords()
-procedure to work correctly, add the wc utility (usually found in
-/usr/bin/wc) to tc.data.
-
-The following program replaces the suffix of the input file (regexp_mapper.words.txt)
-with a new suffix (.count) to create regexp_mapper.words.count.
-
-.regexp_mapper.swift
-*************
-----
-include::../../examples/tutorial/regexp_mapper.swift[]
-----
-**************
-
-fixed_array_mapper
-^^^^^^^^^^^^^^^^^^
-The fixed array mapper maps a list of files into an array. Each
-element of the array is mapped into one file in the specified directory.
-See fixed_array_mapper.swift below.
-
-.fixed_array_mapper.swift
-*************************
-----
-include::../../examples/tutorial/fixed_array_mapper.swift[]
-----
-*************************
-
-foreach
-~~~~~~~
-SwiftScript provides a control structure, foreach, to operate on each
-element of an array in parallel.
-
-In this example, we will run the previous word counting example over
-each file in an array without having to explicitly list the array
-elements. The source code for this example is in foreach.swift. This
-program uses three input files: foreach.1.txt, foreach.2.txt, and
-foreach.3.txt. After you have run the workflow, you should see that there are three
-output files: foreach.1.count, foreach.2.count and foreach.3.count, each
-containing the word count for the corresponding input file. We combine
-the use of the fixed_array_mapper and the regexp_mapper.
-
-.foreach.swift
-**************
-----
-include::../../examples/tutorial/foreach.swift[]
-----
-**************
-
-If
-~~
-Decisions can be made using 'if', like this:
-
-----
-sys::[sed -n '11,15p' ../../examples/tutorial/if.swift]
-----
-
-if.swift contains a simple example of this. Compile and run if.swift
-and see that it outputs "good morning". Changing the morning variable
-from true to false will cause the program to output "good afternoon".
-Here is the contents of the full script:
-
-.if.swift
-*********
------
-include::../../examples/tutorial/if.swift[]
------
-*********
-
-Sequential iteration
-~~~~~~~~~~~~~~~~~~~~
-
-A serial execution of instructions can be carried out using the sequential iteration construct.
-
-The following example demonstrates a simple application. Each step of
-the iteration is a string representation of the byte count of the
-previous step's output, with iteration terminating when the byte count
-reaches zero.
-
-Here's the program:
-
-.sequential_iteration.swift
-***************************
-----
-include::../../examples/tutorial/sequential_iteration.swift[]
-----
-***************************
-
-Echo is the standard unix echo utility.
-
-_wcl_ is our application code. It counts the number of bytes in the one
-file and writes that count out to another, like this:
-
-----
-$ cat ../wcl
-#!/bin/bash
-echo -n $(wc -c < $1) > $2
-
-$ echo -n hello > a
-$ wcl a b
-$ cat b
-5
-----
-
-Install the above wcl script somewhere and add a transformation catalog
-(tc) entry for it (see an example below, note that you will need to change the path in third column to the path where wcl is located on your localhost).
-
-----
-localhost wcl /home/ketan/bin/wcl INSTALLED INTEL32::LINUX null
-----
-
-Then run the example program like this:
-
-
-----
-$ swift iterate.swift
-Swift svn swift-r3334 cog-r2752
-
-RunID: 20100526-2259-gtlz8zf4
-Progress:
-SwiftScript trace: extract int value , 16.0
-SwiftScript trace: extract int value , 2.0
-SwiftScript trace: extract int value , 1.0
-Final status: Finished successfully:4
-----
Copied: trunk/docs/tutorial/language_features (from rev 5078, branches/release-0.93/docs/tutorial/language_features)
===================================================================
--- trunk/docs/tutorial/language_features (rev 0)
+++ trunk/docs/tutorial/language_features 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,390 @@
+Language features
+-----------------
+
+Parameters
+~~~~~~~~~~
+
+Procedures can have parameters. Input parameters specify inputs to the
+procedure and output parameters specify outputs. Our hello world greeting
+procedure already uses an output parameter, t, which indicates where the
+greeting output will go. In this section, we will modify the previous
+script to add an input parameter to the greeting function.
+
+.parameter.swift
+****************
+----
+include::../../examples/tutorial/parameter.swift[]
+----
+****************
+
+We have modified the signature of the greeting procedure to indicate
+that it takes a single parameter, s, of type 'string'.
+
+We have modified the invocation of the 'echo' utility so that it takes
+the value of s as a parameter, instead of the string literal "Hello,
+world!".
+
+We have modified the output file definition to point to a different file
+on disk.
+
+We have modified the invocation of greeting so that a greeting string is
+supplied.
+
+The code for this section can be found in parameter.swift. It can be
+invoked using the swift command, with output appearing in parameter.hello.txt:
+
+
+----
+$ swift parameter.swift
+----
+
+Now that we can choose our greeting text, we can call the same procedure
+with different parameters to generate several output files with
+different greetings. The code is in manyparam.swift and can be run as
+before using the swift command.
+
+.manyparam.swift
+****************
+----
+include::../../examples/tutorial/manyparam.swift[]
+----
+****************
+
+Note that we can intermingle definitions of variables with invocations
+of procedures.
+
+When this program runs, there should be three new files in the
+working directory (manyparam.english.txt, manyparam.francais.txt
+and manyparam.nihongo.txt) each containing a greeting in a different
+language.
+
+In addition to specifying parameters positionally, parameters can be
+named, and if desired a default value can be specified.
+
+Adding another application
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+Now we'll define a new application procedure. The procedure we define
+will capitalise all the words in the input file.
+
+To do this, we'll use the unix tr (translate) utility. Here is an
+example of using tr on the unix command line, not using Swift:
+
+----
+$ echo hello | tr '[a-z]' '[A-Z]'
+HELLO
+----
+
+There are two main steps - updating the transformation catalog, and
+updating the application block.
+
+The transformation catalog lists where application executables are located on
+remote sites. We need to modify the transformation catalog to define a logical
+transformation for the tr utility. The transformation catalog can be found
+in etc/tc.data. There are already several entries specifying
+where executables can be found. Add a new line to the file, specifying
+where tr can be found (usually in /usr/bin/tr but it may differ on
+your system), like this:
+
+----
+localhost tr /usr/bin/tr INSTALLED INTEL32::LINUX null
+----
+
+For now, ignore all of the fields except the second and the third. The
+second field tr specifies a logical application name and the third
+specifies the location of the application executable.
+
+Now that we have defined where to find tr, we can use it in SwiftScript.
+
+We can define a new procedure, capitalise, which calls tr.
+
+----
+sys::[sed -n '7,9p' ../../examples/tutorial/capitalise.swift]
+----
+We can call capitalise like this:
+
+----
+sys::[sed -n '12,14p' ../../examples/tutorial/capitalise.swift]
+----
+
+Here is the full program based on this exercise:
+
+.capitalise.swift
+***********************
+----
+include::../../examples/tutorial/capitalise.swift[]
+----
+***********************
+
+Next, run swift and verify the output is correct.
+
+----
+$ swift capitalise.swift
+...
+$ cat capitalise.2.txt
+HELLO FROM SWIFT
+----
+
+Anonymous files
+~~~~~~~~~~~~~~~
+In the previous section, the file hello.txt is used only to store
+an intermediate result. We don't really care about which name is used
+for the file, and we can let Swift choose the name.
+
+To do that, omit the mapping entirely when declaring hellofile:
+
+----
+sys::[sed -n '11p' ../../examples/tutorial/capitalise_anonymous.swift]
+----
+
+Swift will choose a filename, which in the present version will be in a
+subdirectory called _concurrent.
+
+
+Datatypes
+~~~~~~~~~
+All data in variables and files has a data type. So far, we've seen two
+types:
+
+ - string - this is a built-in type for storing strings of text in
+ memory, much like in other programming languages
+ - messagefile - this is a user-defined type used to mark disc resident files as
+ containing messages
+
+SwiftScript has the additional built-in types: boolean, integer and
+float that function much like their counterparts in other programming
+languages.
+
+It is also possible to create user defined types with more structure,
+for example:
+
+
+----
+sys::[sed -n '3,6p' ../../examples/tutorial/types.swift]
+----
+
+Each element of the structured type can be accessed using a . like this:
+
+----
+sys::[sed -n '14p' ../../examples/tutorial/types.swift]
+----
+
+The following complete program, types.swift, outputs a greeting using a
+user-defined structure type to hold parameters for the message:
+
+.types.swift
+************
+----
+include::../../examples/tutorial/types.swift[]
+----
+************
+
+Structured types can be comprised of marker types for files. See the
+later section on mappers for more information about this.
+
+
+Arrays
+~~~~~~
+
+We can define arrays using the [] suffix in a variable declaration:
+
+----
+sys::[sed -n '9p' ../../examples/tutorial/arrays.swift]
+----
+
+This program, arrays.swift, will declare an array of message files.
+
+.arrays.swift
+*************
+----
+include::../../examples/tutorial/arrays.swift[]
+----
+*************
+
+Observe that the type of the parameter to greeting is now an array of
+strings, string s[], instead of a single string, string s, that
+elements of the array can be referenced numerically, for example s[0],
+and that the array is initialised using an array literal,
+["how","are","you"].
+
+
+Mappers
+~~~~~~~
+
+A significant difference between SwiftScript and other languages is that
+data can be referred to on disk through variables in a very similar
+fashion to data in memory. For example, in the above examples we have
+seen a variable definition like this:
+
+----
+sys::[sed -n '7p' ../../examples/tutorial/arrays.swift]
+----
+
+This means that outfile is a dataset variable, which is mapped to a
+file on disk called arrays.txt. This variable can be assigned to
+using = in a similar fashion to an in-memory variable. We can say that
+outfile is mapped onto the disk file arrays.txt by a mapper.
+
+There are various ways of mapping in SwiftScript. Two forms of mapping,
+simple named mapping and anonymous mapping, have already
+been seen in this tutorial. Later exercises will introduce more forms.
+
+In simple named mapping, the name of the file that a variable is mapped to
+is explictly listed.
+
+----
+sys::[sed -n '7p' ../../examples/tutorial/hello.swift]
+----
+
+This is useful when you want to explicitly name input and output files
+for your program. An example of this can be seen with 'outfile' in the
+hello world exercise.
+
+With anonymous mapping no name is specified in the source code. A name is
+automatically generated for the file. This is useful for intermediate
+files that are only referenced through SwiftScript.
+A variable declaration is mapped anonymously by ommitting any mapper
+definition.
+
+----
+sys::[sed -n '11p' ../../examples/tutorial/capitalise_anonymous.swift]
+----
+
+Later exercises will introduce other ways of mapping from disk files to
+SwiftScript variables.
+
+
+The Regexp Mapper
+^^^^^^^^^^^^^^^^^
+In this exercise, we introduce the regexp mapper. This mapper
+transforms a string expression using a regular expression, and uses the
+result of that transformation as the filename to map.
+
+regexp.swift demonstrates the use of this by placing output into a
+file that is based on the name of the input file. Our input file is
+mapped to the inputfile variable using the simple named mapper, then
+we use the regular expression mapper to map the output file. We then use
+the countwords() procedure to count the words in the input file and
+store the result in the output file. In order for the countwords()
+procedure to work correctly, add the wc utility (usually found in
+/usr/bin/wc) to tc.data.
+
+The following program replaces the suffix of the input file (regexp_mapper.words.txt)
+with a new suffix (.count) to create regexp_mapper.words.count.
+
+.regexp_mapper.swift
+*************
+----
+include::../../examples/tutorial/regexp_mapper.swift[]
+----
+**************
+
+fixed_array_mapper
+^^^^^^^^^^^^^^^^^^
+The fixed array mapper maps a list of files into an array. Each
+element of the array is mapped into one file in the specified directory.
+See fixed_array_mapper.swift below.
+
+.fixed_array_mapper.swift
+*************************
+----
+include::../../examples/tutorial/fixed_array_mapper.swift[]
+----
+*************************
+
+foreach
+~~~~~~~
+SwiftScript provides a control structure, foreach, to operate on each
+element of an array in parallel.
+
+In this example, we will run the previous word counting example over
+each file in an array without having to explicitly list the array
+elements. The source code for this example is in foreach.swift. This
+program uses three input files: foreach.1.txt, foreach.2.txt, and
+foreach.3.txt. After you have run the workflow, you should see that there are three
+output files: foreach.1.count, foreach.2.count and foreach.3.count, each
+containing the word count for the corresponding input file. We combine
+the use of the fixed_array_mapper and the regexp_mapper.
+
+.foreach.swift
+**************
+----
+include::../../examples/tutorial/foreach.swift[]
+----
+**************
+
+If
+~~
+Decisions can be made using 'if', like this:
+
+----
+sys::[sed -n '11,15p' ../../examples/tutorial/if.swift]
+----
+
+if.swift contains a simple example of this. Compile and run if.swift
+and see that it outputs "good morning". Changing the morning variable
+from true to false will cause the program to output "good afternoon".
+Here is the contents of the full script:
+
+.if.swift
+*********
+-----
+include::../../examples/tutorial/if.swift[]
+-----
+*********
+
+Sequential iteration
+~~~~~~~~~~~~~~~~~~~~
+
+A serial execution of instructions can be carried out using the sequential iteration construct.
+
+The following example demonstrates a simple application. Each step of
+the iteration is a string representation of the byte count of the
+previous step's output, with iteration terminating when the byte count
+reaches zero.
+
+Here's the program:
+
+.sequential_iteration.swift
+***************************
+----
+include::../../examples/tutorial/sequential_iteration.swift[]
+----
+***************************
+
+Echo is the standard unix echo utility.
+
+_wcl_ is our application code. It counts the number of bytes in the one
+file and writes that count out to another, like this:
+
+----
+$ cat ../wcl
+#!/bin/bash
+echo -n $(wc -c < $1) > $2
+
+$ echo -n hello > a
+$ wcl a b
+$ cat b
+5
+----
+
+Install the above wcl script somewhere and add a transformation catalog
+(tc) entry for it (see an example below, note that you will need to change the path in third column to the path where wcl is located on your localhost).
+
+----
+localhost wcl /home/ketan/bin/wcl INSTALLED INTEL32::LINUX null
+----
+
+Then run the example program like this:
+
+
+----
+$ swift iterate.swift
+Swift svn swift-r3334 cog-r2752
+
+RunID: 20100526-2259-gtlz8zf4
+Progress:
+SwiftScript trace: extract int value , 16.0
+SwiftScript trace: extract int value , 2.0
+SwiftScript trace: extract int value , 1.0
+Final status: Finished successfully:4
+----
Deleted: trunk/docs/tutorial/runtime_features
===================================================================
--- trunk/docs/tutorial/runtime_features 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/tutorial/runtime_features 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,317 +0,0 @@
-Runtime features
-----------------
-
-Visualizing the workflow as a graph
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-When running a workflow, its possible to generate a provenance graph at
-the same time:
-
-
------
-$ swift -pgraph graph.dot first.swift
-$ dot -ograph.png -Tpng graph.dot
-----
-
-graph.png can then be viewed using your favourite image viewer. The dot application is part of the graphViz project. More information can be found at http://www.graphviz.org.
-
-Running on a remote site
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-As configured by default, all jobs are run locally. In the previous
-examples, we've invoked echo and tr executables from our SwiftScript
-program. These have been run on the local system (the same computer on
-which you ran swift). We can also make our computations run on a
-remote resource.
-
-WARNING: This example is necessarily more vague than previous ones,
-because it requires access to remote resources. You should ensure that
-you can submit a job using the globus-job-run (or globusrun-ws)
-command(s).
-
-We do not need to modify any SwiftScript code to run on another
-resource. Instead, we must modify another catalog, the site catalog.
-This catalog provides details of the location that applications will be
-run, with the default settings referring to the local machine. We will
-modify it to refer to a remote resource - the UC Teraport cluster. If
-you are not a UC Teraport user, you should use details of a different
-resource that you do have access to.
-
-The site catalog is located in etc/sites.xml and is a relatively
-straightforward XML format file. We must modify each of the following
-three settings: gridftp (which indicates how and where data can be
-transferred to the remote resource), jobmanager (which indicates how
-applications can be run on the remote resource) and workdirectory (which
-indicates where working storage can be found on the remote resource).
-
-
-Writing a mapper
-~~~~~~~~~~~~~~~~
-
-This section will introduce writing a custom mapper so that Swift is
-able to access data files laid out in application-specific ways.
-
-An application-specific mapper must take the form of a Java class that
-implements the http://www.ci.uchicago.edu/swift/javadoc/vdsk/org/griphyn/vdl/mapping/Mapper.html[Mapper
-interface].
-
-Usually you don't need to implement this interface directly, because
-Swift provides a number of more concrete classes with some functionality
-already implemented.
-
-The hierarchy of helper classes is:
-
-http://www.ci.uchicago.edu/swift/javadoc/vdsk/org/griphyn/vdl/mapping/Mapper.html[Mapper]
-
-This is the abstract interface for mappers in Swift. You must
-implement methods to provide access to mapper properties, to map from a
-SwiftScript dataset path (such as foo[1].bar) to a file name, to check
-whether a file exists. None of the default Swift mappers implement this
-interface directly - instead they use one of the following helper classes.
-
-http://www.ci.uchicago.edu/swift/javadoc/vdsk/org/griphyn/vdl/mapping/AbstractMapper.html[Abstract Mapper]
-
-This provides helper methods to manage mapper properties and to handle
-existance checking. Examples of mappers which use this class are:
-http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.array_mapper[Array Mapper],
-http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.csv_mapper[CSV Mapper],
-http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.fixed_array_mapper[Fixed Array Mapper],
-http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.regexp_mapper[Regexp Mapper],
-and http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.single_file_mapper[Single File Mapper].
-
-http://www.ci.uchicago.edu/swift/javadoc/vdsk/org/griphyn/vdl/mapping/file/AbstractFileMapper.html[AbstractFileMapper]
-This provides a helper class for mappers which select files based on
-selecting files from a directory listing. It is necessary to write some
-helper methods that are different from the above mapper methods.
-Examples of mappers which use this class are:
-http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.simple_mapper[simple_mapper],
-http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.filesys_mapper[filesys_mapper],
-and the StructuredRegularExpressionMapper.
-
-In general, to write a mapper, choose either the AbstractMapper or the
-AbstractFileMapper and extend those. If your mapper will generally
-select the files it returns based on a directory listing and will
-convert paths to filenames using some regular conversion (for example,
-in the way that simple_mapper maps files in a directory that match a
-particular pattern), then you should probably use the
-AbstractFileMapper. If your mapper will produce a list of files in some
-other way (for example, in the way that csv_mapper maps based on
-filenames given in a CSV file rather than looking at which files are in
-a directory), then you should probably use the AbstractMapper.
-
-
-Writing a very basic mapper
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-In this section, we will write a very basic (almost useless) mapper that
-will map a SwiftScript dataset into a hardcoded file called
-myfile.txt, like this:
-
-
- Swift variable Filename
-
- var <-----------------------------> myfile.txt
-
-We should be able to use the mapper we write in a SwiftScript program
-like this:
-
-
-----
-type file;
-file f <my_first_mapper>;
-----
-
-First we must choose a base class - AbstractMapper or
-AbstractFileMapper. We aren't going to use a directory listing to decide
-on our mapping - we are getting the mapping from some other source (in
-fact, it will be hard coded). So we will use AbstractMapper.
-
-So now onto the source code. We must define a subclass of AbstractMapper
-and implement several mapper methods: isStatic, existing, and map. These
-methods are documented in the javadoc for the Mapper interface.
-
-Here is the code implementing this mapper. Put this in your source
-vdsk directory, make a directory src/tutorial/|and put this file in
-src/tutorial/MyFirstMapper.java.
-
-.MyFirstMapper.java
-*******************
-----
-include::../../examples/tutorial/MyFirstMapper.java[]
-----
-*******************
-
-Now we need to inform the Swift engine about the existence of this
-mapper. We do that by editing the MapperFactory class definition, in
-src/org/griphyn/vdl/mapping/MapperFactory.java and adding a
-registerMapper call alongside the existing registerMapper calls, like this:
-
-
-----
-registerMapper("my_first_mapper", tutorial.MyFirstMapper.class);
-----
-
-The first parameter is the name of the mapper that will be used in
-SwiftScript program. The second parameter is the new Mapper class that
-we just wrote.
-
-Now rebuild Swift using the "ant redist" target.
-
-This new Swift build will be aware of your new mapper. We can test it
-out with a hello world program:
-
-.my_first_mapper.swift
-**********************
------
-include::../../examples/tutorial/my_first_mapper.swift[]
------
-**********************
-
-Run this program, and hopefully you will find the "hello" string has
-been output into the hard coded output file myfile.txt:
-
-
-----
-$ cat myfile.txt
-hello
-----
-
-So that's a first very simple mapper implemented. Compare the source
-code to the single_file_mapper in
-http://www.ci.uchicago.edu/trac/swift/browser/trunk/src/org/griphyn/vdl/mapping/file/SingleFileMapper.java[SingleFileMapper.java].
-There is not much more code to the single_file_mapper - mostly code to
-deal with the file parameter.
-
-
-Starting and restarting
-~~~~~~~~~~~~~~~~~~~~~~~
-
-Now we're going to try out the restart capabilities of Swift. We will
-make a workflow that will deliberately fail, and then we will fix the
-problem so that Swift can continue with the workflow.
-
-First we have the program in working form, restart.swift.
-
-.restart.swift
-**************
-----
-include::../../examples/tutorial/restart.swift[]
-----
-**************
-
-We must define some transformation catalog entries:
-
-----
-localhost touch /usr/bin/touch INSTALLED INTEL32::LINUX null
-localhost broken /bin/true INSTALLED INTEL32::LINUX null
-----
-
-Now we can run the program:
-
-----
-$ swift restart.swift
-Swift 0.9 swift-r2860 cog-r2388
-
-RunID: 20100526-1119-3kgzzi15
-Progress:
-Final status: Finished successfully:4
-----
-
-Four jobs run - touch, echo, broken and a final echo. (note that broken
-isn't actually broken yet).
-
-Now we will break the broken job and see what happens. Replace the
-definition in tc.data for broken with this:
-
-----
-localhost broken /bin/false INSTALLED INTEL32::LINUX null
-----
-
-Now when we run the workflow, the broken task fails:
-
-----
-$ swift restart.swift
-
-Swift 0.9 swift-r2860 cog-r2388
-
-RunID: 20100526-1121-tssdcljg
-Progress:
-Progress: Stage in:1 Finished successfully:2
-Execution failed:
- Exception in broken:
-Arguments: [process]
-Host: localhost
-Directory: restart-20100526-1121-tssdcljg/jobs/1/broken-1i6ufisj
-stderr.txt:
-stdout.txt:
-----
-
-From the output we can see that touch and the first echo completed, but
-then broken failed and so swift did not attempt to execute the final echo.
-
-There will be a restart log with the same name as the RunID:
-
-
-----
-$ ls *20100526-1121-tssdcljg*rlog
-restart-20100526-1121-tssdcljg.0.rlog
-----
-
-This restart log contains enough information for swift to know which
-parts of the workflow were executed successfully.
-
-We can try to rerun it immediately, like this:
-
-
-----
-$ swift -resume restart-20100526-1121-tssdcljg.0.rlog restart.swift
-
-Swift 0.9 swift-r2860 cog-r2388
-
-RunID: 20100526-1125-7yx0zi6d
-Progress:
-Execution failed:
- Exception in broken:
-Arguments: [process]
-Host: localhost
-Directory: restart-20100526-1125-7yx0zi6d/jobs/m/broken-msn1gisj
-stderr.txt:
-stdout.txt:
-
-Caused by:
- Exit code 1
-----
-
-Swift tried to resume the workflow by executing "broken" again. It did
-not try to run the touch or first echo jobs, because the restart log
-says that they do not need to be executed again.
-
-Broken failed again, leaving the original restart log in place.
-
-Now we will fix the problem with "broken" by restoring the original
-tc.data line that works.
-
-Remove the existing "broken" line and replace it with the successful
-tc.data entry above:
-
-
-----
-localhost broken /bin/true INSTALLED INTEL32::LINUX null
-----
-
-Now run again:
-
-
-----
-$ swift -resume restart-20100526-1121-tssdcljg.0.rlog restart.swift
-
-Swift 0.9 swift-r2860 cog-r2388
-
-RunID: 20100526-1128-a2gfuxhg
-Progress:
-Final status: Initializing:2 Finished successfully:2
-----
-
-Swift tries to run "broken" again. This time it works, and so Swift
-continues on to execute the final piece of the workflow as if nothing
-had ever gone wrong.
Copied: trunk/docs/tutorial/runtime_features (from rev 5078, branches/release-0.93/docs/tutorial/runtime_features)
===================================================================
--- trunk/docs/tutorial/runtime_features (rev 0)
+++ trunk/docs/tutorial/runtime_features 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,160 @@
+Runtime features
+----------------
+
+Visualizing the workflow as a graph
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When running a workflow, its possible to generate a provenance graph at
+the same time:
+
+
+-----
+$ swift -pgraph graph.dot first.swift
+$ dot -ograph.png -Tpng graph.dot
+----
+
+graph.png can then be viewed using your favourite image viewer. The dot application is part of the graphViz project. More information can be found at http://www.graphviz.org.
+
+Running on a remote site
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+As configured by default, all jobs are run locally. In the previous
+examples, we've invoked echo and tr executables from our SwiftScript
+program. These have been run on the local system (the same computer on
+which you ran swift). We can also make our computations run on a
+remote resource. For more information on running Swift on a remote site
+please see the http://www.ci.uchicago.edu/swift/guides/release-0.93/siteguide/siteguide.html[Site Configuration Guide].
+
+
+Starting and restarting
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Now we're going to try out the restart capabilities of Swift. We will
+make a workflow that will deliberately fail, and then we will fix the
+problem so that Swift can continue with the workflow.
+
+First we have the program in working form, restart.swift.
+
+.restart.swift
+**************
+----
+include::../../examples/tutorial/restart.swift[]
+----
+**************
+
+We must define some transformation catalog entries:
+
+----
+localhost touch /usr/bin/touch INSTALLED INTEL32::LINUX null
+localhost broken /bin/true INSTALLED INTEL32::LINUX null
+----
+
+Now we can run the program:
+
+----
+$ swift restart.swift
+Swift 0.9 swift-r2860 cog-r2388
+
+RunID: 20100526-1119-3kgzzi15
+Progress:
+Final status: Finished successfully:4
+----
+
+Four jobs run - touch, echo, broken and a final echo. (note that broken
+isn't actually broken yet).
+
+Now we will break the broken job and see what happens. Replace the
+definition in tc.data for broken with this:
+
+----
+localhost broken /bin/false INSTALLED INTEL32::LINUX null
+----
+
+Now when we run the workflow, the broken task fails:
+
+----
+$ swift restart.swift
+
+Swift 0.9 swift-r2860 cog-r2388
+
+RunID: 20100526-1121-tssdcljg
+Progress:
+Progress: Stage in:1 Finished successfully:2
+Execution failed:
+ Exception in broken:
+Arguments: [process]
+Host: localhost
+Directory: restart-20100526-1121-tssdcljg/jobs/1/broken-1i6ufisj
+stderr.txt:
+stdout.txt:
+----
+
+From the output we can see that touch and the first echo completed, but
+then broken failed and so swift did not attempt to execute the final echo.
+
+There will be a restart log with the same name as the RunID:
+
+
+----
+$ ls *20100526-1121-tssdcljg*rlog
+restart-20100526-1121-tssdcljg.0.rlog
+----
+
+This restart log contains enough information for swift to know which
+parts of the workflow were executed successfully.
+
+We can try to rerun it immediately, like this:
+
+
+----
+$ swift -resume restart-20100526-1121-tssdcljg.0.rlog restart.swift
+
+Swift 0.9 swift-r2860 cog-r2388
+
+RunID: 20100526-1125-7yx0zi6d
+Progress:
+Execution failed:
+ Exception in broken:
+Arguments: [process]
+Host: localhost
+Directory: restart-20100526-1125-7yx0zi6d/jobs/m/broken-msn1gisj
+stderr.txt:
+stdout.txt:
+
+Caused by:
+ Exit code 1
+----
+
+Swift tried to resume the workflow by executing "broken" again. It did
+not try to run the touch or first echo jobs, because the restart log
+says that they do not need to be executed again.
+
+Broken failed again, leaving the original restart log in place.
+
+Now we will fix the problem with "broken" by restoring the original
+tc.data line that works.
+
+Remove the existing "broken" line and replace it with the successful
+tc.data entry above:
+
+
+----
+localhost broken /bin/true INSTALLED INTEL32::LINUX null
+----
+
+Now run again:
+
+
+----
+$ swift -resume restart-20100526-1121-tssdcljg.0.rlog restart.swift
+
+Swift 0.9 swift-r2860 cog-r2388
+
+RunID: 20100526-1128-a2gfuxhg
+Progress:
+Final status: Initializing:2 Finished successfully:2
+----
+
+Swift tries to run "broken" again. This time it works, and so Swift
+continues on to execute the final piece of the workflow as if nothing
+had ever gone wrong.
Deleted: trunk/docs/tutorial/tutorial.txt
===================================================================
--- trunk/docs/tutorial/tutorial.txt 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/tutorial/tutorial.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,17 +0,0 @@
-A Swift Tutorial
-================
-
-:toc:
-:icons:
-:website: http://www.ci.uchicago.edu/swift/guides/tutorial.php
-:numbered:
-
-include::introduction[]
-
-include::hello_world[]
-
-include::language_features[]
-
-include::runtime_features[]
-
-include::bits[]
Copied: trunk/docs/tutorial/tutorial.txt (from rev 5078, branches/release-0.93/docs/tutorial/tutorial.txt)
===================================================================
--- trunk/docs/tutorial/tutorial.txt (rev 0)
+++ trunk/docs/tutorial/tutorial.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,17 @@
+A Swift Tutorial
+================
+
+:toc:
+:icons:
+:website: http://www.ci.uchicago.edu/swift/guides/tutorial.php
+:numbered:
+
+include::introduction[]
+
+include::hello_world[]
+
+include::language_features[]
+
+include::runtime_features[]
+
+include::bits[]
Modified: trunk/docs/userguide/cdm
===================================================================
--- trunk/docs/userguide/cdm 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/userguide/cdm 2011-09-07 16:32:23 UTC (rev 5079)
@@ -2,202 +2,202 @@
--------------------------
Overview
-~~~~~~~~
-. The user specifies a CDM policy in a file, customarily fs.data.
-. fs.data is given to Swift on the command line.
-. The Swift data module (org.globus.swift.data) is informed of the CDM policy.
-. At job launch time, the VDL Karajan code queries the CDM policy,
- .. altering the file staging phase, and
- .. sending fs.data to the compute site.
-. At job run time, the Swift wrapper script
- .. consults a Perl script to obtain policy, and
- .. uses wrapper extensions to modify data movement.
-. Similarly, stage out can be changed.
-
-
+~~~~~~~~
+. The user specifies a CDM policy in a file, customarily fs.data.
+. fs.data is given to Swift on the command line.
+. The Swift data module (org.globus.swift.data) is informed of the CDM policy.
+. At job launch time, the VDL Karajan code queries the CDM policy,
+ .. altering the file staging phase, and
+ .. sending fs.data to the compute site.
+. At job run time, the Swift wrapper script
+ .. consults a Perl script to obtain policy, and
+ .. uses wrapper extensions to modify data movement.
+. Similarly, stage out can be changed.
+
+
.Command line
+-----
+$ swift -sites.file sites.xml -tc.file tc.data -cdm.file fs.data stream.swift
-----
-$ swift -sites.file sites.xml -tc.file tc.data -cdm.file fs.data stream.swift
------
-
+
CDM policy file format
~~~~~~~~~~~~~~~~~~~~~~
.Example
+-----
+# Describe CDM for my job
+property GATHER_LIMIT 1
+rule .*input.txt DIRECT /gpfs/homes/wozniak/data
+rule .*xfile*.data BROADCAST /dev/shm
+rule .* DEFAULT
-----
-# Describe CDM for my job
-property GATHER_LIMIT 1
-rule .*input.txt DIRECT /gpfs/homes/wozniak/data
-rule .*xfile*.data BROADCAST /dev/shm
-rule .* DEFAULT
------
-
-The lines contain:
-
-. A directive, either rule or property
-. A rule has:
- .. A regular expression
- .. A policy token
- .. Additional policy-specific arguments
-. A property has
- .. A policy property token
- .. The token value
-. Comments with # .
-. Blank lines are ignored.
-
-
+
+The lines contain:
+
+. A directive, either rule or property
+. A rule has:
+ .. A regular expression
+ .. A policy token
+ .. Additional policy-specific arguments
+. A property has
+ .. A policy property token
+ .. The token value
+. Comments with # .
+. Blank lines are ignored.
+
+
.Notes
-
-. The policy file is used as a lookup database by Swift and Perl methods.
-. For example, a lookup with the database above given the argument input.txt would result in the Direct policy.
-. If the lookup does not succeed, the result is DEFAULT.
- . Policies are listed as subclasses of org.globus.swift.data.policy.Policy .
-
-
+
+. The policy file is used as a lookup database by Swift and Perl methods.
+. For example, a lookup with the database above given the argument input.txt would result in the Direct policy.
+. If the lookup does not succeed, the result is DEFAULT.
+ . Policies are listed as subclasses of org.globus.swift.data.policy.Policy .
+
+
Policy Descriptions
~~~~~~~~~~~~~~~~~~~
-.Default
-
-* Just use file staging as provided by Swift/CoG. Identical to behavior if no CDM file is given.
-
-
+.Default
+
+* Just use file staging as provided by Swift/CoG. Identical to behavior if no CDM file is given.
+
+
.Broadcast
------
-rule .*xfile*.data BROADCAST /dev/shm
------
-* The input files matching the pattern will be stored in the given directory, an LFS location, with links in the job directory.
-* On the BG/P, this will make use of the f2cn tool.
-* On machines not implementing an efficient broadcast, we will just ensure correctness. For example, on a workstation, the location could be in a /tmp RAM FS.
-
-
+-----
+rule .*xfile*.data BROADCAST /dev/shm
+-----
+* The input files matching the pattern will be stored in the given directory, an LFS location, with links in the job directory.
+* On the BG/P, this will make use of the f2cn tool.
+* On machines not implementing an efficient broadcast, we will just ensure correctness. For example, on a workstation, the location could be in a /tmp RAM FS.
+
+
.Direct
------
-rule .*input.txt DIRECT /gpfs/scratch/wozniak/
------
-* Allows for direct I/O to the parallel FS without staging.
-* The input files matching the pattern must already exist in the given directory, a GFS location. Links will be placed in the job directory.
-* The output files matching the pattern will be stored in the given directory, with links in the job directory.
-* Example: In the rule above, the Swift-generated file name ./data/input.txt would be accessed by the user job in /gpfs/homes/wozniak/data/input.txt .
-
-
-.Local
------
-rule .*input.txt LOCAL dd /gpfs/homes/wozniak/data obs=64K
------
-* Allows for client-directed input copy to the compute node.
-* The user may specify cp or dd as the input transfer program.
-* The input files matching the pattern must already exist in the given directory, a GFS location. Copies will be placed in the job directory.
-* Argument list: [tool] [GFS directory] [tool arguments]*
-
-
+-----
+rule .*input.txt DIRECT /gpfs/scratch/wozniak/
+-----
+* Allows for direct I/O to the parallel FS without staging.
+* The input files matching the pattern must already exist in the given directory, a GFS location. Links will be placed in the job directory.
+* The output files matching the pattern will be stored in the given directory, with links in the job directory.
+* Example: In the rule above, the Swift-generated file name ./data/input.txt would be accessed by the user job in /gpfs/homes/wozniak/data/input.txt .
+
+
+.Local
+-----
+rule .*input.txt LOCAL dd /gpfs/homes/wozniak/data obs=64K
+-----
+* Allows for client-directed input copy to the compute node.
+* The user may specify cp or dd as the input transfer program.
+* The input files matching the pattern must already exist in the given directory, a GFS location. Copies will be placed in the job directory.
+* Argument list: [tool] [GFS directory] [tool arguments]*
+
+
.Gather
+-----
+property GATHER_LIMIT 500000000 # 500 MB
+property GATHER_DIR /dev/shm/gather
+property GATHER_TARGET /gpfs/wozniak/data/gather_target
+rule .*.output.txt GATHER
-----
-property GATHER_LIMIT 500000000 # 500 MB
-property GATHER_DIR /dev/shm/gather
-property GATHER_TARGET /gpfs/wozniak/data/gather_target
-rule .*.output.txt GATHER
------
-
-* The output files matching the pattern will be present to tasks in the job directory as usual but noted in a _swiftwrap shell array GATHER_OUTPUT.
-* The GATHER_OUTPUT files will be cached in the GATHER_DIR, an LFS location.
-* The cache will be flushed when a job ends if a du on GATHER_DIR exceeds GATHER_LIMIT.
-* As the cache fills or on stage out, the files will be bundled into randomly named tarballs in GATHER_TARGET, a GFS location.
-* If the compute node is an SMP, GATHER_DIR is a shared resource. It is protected by the link file GATHER_DIR/.cdm.lock .
-* Unpacking the tarballs in GATHER_TARGET will produce the user-specified filenames.
-
+
+* The output files matching the pattern will be present to tasks in the job directory as usual but noted in a _swiftwrap shell array GATHER_OUTPUT.
+* The GATHER_OUTPUT files will be cached in the GATHER_DIR, an LFS location.
+* The cache will be flushed when a job ends if a du on GATHER_DIR exceeds GATHER_LIMIT.
+* As the cache fills or on stage out, the files will be bundled into randomly named tarballs in GATHER_TARGET, a GFS location.
+* If the compute node is an SMP, GATHER_DIR is a shared resource. It is protected by the link file GATHER_DIR/.cdm.lock .
+* Unpacking the tarballs in GATHER_TARGET will produce the user-specified filenames.
+
.Summary
-
-. Files created by application
-. Acquire lock
-. Move files to cache
-. Check cache size
-. If limit exceeded, move all cache files to outbox
-. Release lock
-. If limit was exceeded, stream outbox as tarball to target
-
+
+. Files created by application
+. Acquire lock
+. Move files to cache
+. Check cache size
+. If limit exceeded, move all cache files to outbox
+. Release lock
+. If limit was exceeded, stream outbox as tarball to target
+
.Notes
-
-* Gather required quite a bit of shell functionality to manage the lock, etc. This is placed in cdm_lib.sh .
-* vdl_int.k needed an additional task submission (cdm_cleanup.sh) to perform the final flush at workflow completion time . This task also uses cdm_lib.sh .
-
-
+
+* Gather required quite a bit of shell functionality to manage the lock, etc. This is placed in cdm_lib.sh .
+* vdl_int.k needed an additional task submission (cdm_cleanup.sh) to perform the final flush at workflow completion time . This task also uses cdm_lib.sh .
+
+
VDL/Karajan processing
~~~~~~~~~~~~~~~~~~~~~~
-. CDM functions are available in Karajan via the cdm namespace.
-. These functions are defined in org.globus.swift.data.Query .
-. If CDM is enabled, VDL skips file staging for files unless the policy is DEFAULT.
-
-
-Swift wrapper CDM routines
+. CDM functions are available in Karajan via the cdm namespace.
+. These functions are defined in org.globus.swift.data.Query .
+. If CDM is enabled, VDL skips file staging for files unless the policy is DEFAULT.
+
+
+Swift wrapper CDM routines
~~~~~~~~~~~~~~~~~~~~~~~~~~
-. The cdm.pl script is shipped to the compute node if CDM is enabled.
-. When linking in inputs, CDM is consulted by _swiftwrap:cdm_lookup().
-. The cdm_action() shell function handles CDM methods, typically just producing a link.
-
-
-Test cases
+. The cdm.pl script is shipped to the compute node if CDM is enabled.
+. When linking in inputs, CDM is consulted by _swiftwrap:cdm_lookup().
+. The cdm_action() shell function handles CDM methods, typically just producing a link.
+
+
+Test cases
~~~~~~~~~~
-
-. Simple test cases are in:
- https://svn.mcs.anl.gov/repos/wozniak/collab/cdm/scripts/cdm-direct and
- https://svn.mcs.anl.gov/repos/wozniak/collab/cdm/scripts/all-pairs
-. Do a:
- mkdir cdm
- cd cdm
- svn co https://svn.mcs.anl.gov/repos/wozniak/collab/cdm/scripts
-. In cdm-direct, run:
- source ./setup.sh local local local
-. Run workflow:
- swift -sites.file sites.xml -tc.file tc.data -cdm.file fs.data stream.swift
-. Note that staging is skipped for input.txt
- policy: file://localhost/input.txt : DIRECT
- FILE_STAGE_IN_START file=input.txt ...
- FILE_STAGE_IN_SKIP file=input.txt policy=DIRECT
- FILE_STAGE_IN_END file=input.txt ...
-. In the wrapper output, the input file is handled by CDM functionality:
- Progress 2010-01-21 13:50:32.466572727-0600 LINK_INPUTS
- CDM_POLICY: DIRECT /homes/wozniak/cdm/scripts/cdm-direct
- CDM: jobs/t/cp_sh-tkul4nmj input.txt DIRECT /homes/wozniak/cdm/scripts/cdm-direct
- CDM[DIRECT]: Linking jobs/t/cp_sh-tkul4nmj/input.txt to /homes/wozniak/cdm/scripts/cdm-direct/input.txt
- Progress 2010-01-21 13:50:32.486016708-0600 EXECUTE
-. all-pairs is quite similar but uses more policies.
-
-
+
+. Simple test cases are in:
+ https://svn.mcs.anl.gov/repos/wozniak/collab/cdm/scripts/cdm-direct and
+ https://svn.mcs.anl.gov/repos/wozniak/collab/cdm/scripts/all-pairs
+. Do a:
+ mkdir cdm
+ cd cdm
+ svn co https://svn.mcs.anl.gov/repos/wozniak/collab/cdm/scripts
+. In cdm-direct, run:
+ source ./setup.sh local local local
+. Run workflow:
+ swift -sites.file sites.xml -tc.file tc.data -cdm.file fs.data stream.swift
+. Note that staging is skipped for input.txt
+ policy: file://localhost/input.txt : DIRECT
+ FILE_STAGE_IN_START file=input.txt ...
+ FILE_STAGE_IN_SKIP file=input.txt policy=DIRECT
+ FILE_STAGE_IN_END file=input.txt ...
+. In the wrapper output, the input file is handled by CDM functionality:
+ Progress 2010-01-21 13:50:32.466572727-0600 LINK_INPUTS
+ CDM_POLICY: DIRECT /homes/wozniak/cdm/scripts/cdm-direct
+ CDM: jobs/t/cp_sh-tkul4nmj input.txt DIRECT /homes/wozniak/cdm/scripts/cdm-direct
+ CDM[DIRECT]: Linking jobs/t/cp_sh-tkul4nmj/input.txt to /homes/wozniak/cdm/scripts/cdm-direct/input.txt
+ Progress 2010-01-21 13:50:32.486016708-0600 EXECUTE
+. all-pairs is quite similar but uses more policies.
+
+
PTMap case
-^^^^^^^^^^
-. Start with vanilla PTMap:
- .. cd cdm
- .. mkdir apps
- .. cd apps
- .. https://svn.mcs.anl.gov/repos/wozniak/collab/cdm/apps/ptmap
-. Source setup.sh
-. Use start.sh, which
- .. applies CDM policy from fs.local.data
-
-
+^^^^^^^^^^
+. Start with vanilla PTMap:
+ .. cd cdm
+ .. mkdir apps
+ .. cd apps
+ .. https://svn.mcs.anl.gov/repos/wozniak/collab/cdm/apps/ptmap
+. Source setup.sh
+. Use start.sh, which
+ .. applies CDM policy from fs.local.data
+
+
CDM site-aware policy file format
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
+
Example
-----
-#Describe CDM for my job
-#Use DIRECT and BROADCAST if on cluster1, else use DEFAULT behavior
-property GATHER_LIMIT 1
-rule cluster1 .*input.txt DIRECT /gpfs/homes/wozniak/data
-rule cluster1 .*xfile*.data BROADCAST /dev/shm
-rule ANYWHERE .* DEFAULT
+#Describe CDM for my job
+#Use DIRECT and BROADCAST if on cluster1, else use DEFAULT behavior
+property GATHER_LIMIT 1
+rule cluster1 .*input.txt DIRECT /gpfs/homes/wozniak/data
+rule cluster1 .*xfile*.data BROADCAST /dev/shm
+rule ANYWHERE .* DEFAULT
-----
-
+
The lines contain:
-
-. A directive, either rule or property
-. A rule has:
- .. A regular expression for site matchin
- .. A regular expression for filename matching
- .. A policy token
- .. Additional policy-specific arguments
-. A property has
- .. A policy property token
- .. The token value
-. Comments with # .
-. Blank lines are ignored.
+
+. A directive, either rule or property
+. A rule has:
+ .. A regular expression for site matchin
+ .. A regular expression for filename matching
+ .. A policy token
+ .. Additional policy-specific arguments
+. A property has
+ .. A policy property token
+ .. The token value
+. Comments with # .
+. Blank lines are ignored.
Modified: trunk/docs/userguide/coasters
===================================================================
--- trunk/docs/userguide/coasters 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/userguide/coasters 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,7 +1,5 @@
Coasters
--------
-Coasters were introduced in Swift v0.6 as an experimental feature.
-
In many applications, Swift performance can be greatly enhanced by the
use of CoG coasters. CoG coasters provide a low-overhead job submission
and file transfer mechanism suited for the execution of short jobs (on
Copied: trunk/docs/userguide/log-processing (from rev 5078, branches/release-0.93/docs/userguide/log-processing)
===================================================================
--- trunk/docs/userguide/log-processing (rev 0)
+++ trunk/docs/userguide/log-processing 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,104 @@
+
+Log Processing
+--------------
+
+To properly generate log plots, you must enable VDL/Karajan logging.
+TODO:How?
+
+
+You should check the scripts that you intend to use to determine
+what log lines they require and ensure that you are generating
+those lines via log4j.properties
+
+Make sure log4.properties contains:
+--------------------------------------
+log4j.logger.swift=DEBUG
+log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Cpu=DEBUG
+log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Block=DEBUG
+--------------------------------------
+TODO: Does it work for coasters-based runs only?
+
+Normalize event times in the log to the run start time
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+* Generate the log, assuming the log is titled +swift-run.log+
+
+------------------------------------------
+./normalize-log.pl file.contains.start.time swift-run.log > swift-run.norm
+------------------------------------------
+TODO:In what format does the start time be in 'file.contains.start.time'
+
+
+Make a basic load plot from Coasters Cpu log lines
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+. Normalize the log.
+. Build up a load data file:
++
+------------------------------------------
+./cpu-job-load.pl < swift-run.norm > load.data
+------------------------------------------
+. Plot with the JFreeChart-based plotter in usertools/plotter:
++
+------------------------------------------
+swift_plotter.zsh -s load.cfg load.eps load.data
+------------------------------------------
+Note: Th load.cfg is available from swift/libexec/log-processing/
+
+
+Make a basic job completion plot from Coasters Cpu log lines
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+. Normalize the log.
+
+. Build up a completed data file:
++
+------------------------------------------
+./cpu-job-completed.pl < swift-run.norm > completed.data
+------------------------------------------
+
+. Plot with the JFreeChart-based plotter in usertools/plotter:
++
+------------------------------------------
+swift_plotter.zsh -s completed.cfg completed.eps completed.data
+------------------------------------------
+
+Make a basic Block allocation plot from Coasters Block log lines
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+. Normalize the log.
+
+. Build up a block allocation data file:
++
+------------------------------------------
+./block-level.pl < swift-run.norm > blocks.data
+------------------------------------------
+
+. Plot with the JFreeChart-based plotter in usertools/plotter:
++
+------------------------------------------
+swift_plotter.zsh -s blocks.{cfg,eps,data}
+------------------------------------------
+
+Make a job runtime distribution plot from Coasters Cpu log lines
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+. Normalize the log.
+
+. Build up a job runtime file:
++
+------------------------------------------
+./extract-times.pl < swift-run.norm > times.data
+------------------------------------------
+
+. Put the job runtimes into 1-second buckets:
++
+------------------------------------------
+./ buckets.pl 1 times.data > buckets.data
+------------------------------------------
+
+. Plot with the JFreeChart-based plotter in usertools/plotter:
++
+------------------------------------------
+swift_plotter.zsh -s buckets.cfg buckets.eps buckets.data
+------------------------------------------
Modified: trunk/docs/userguide/userguide.txt
===================================================================
--- trunk/docs/userguide/userguide.txt 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/userguide/userguide.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -37,3 +37,5 @@
include::howto_tips[]
include::cdm[]
+
+include::log-processing[]
Deleted: trunk/docs/utils/gensites.txt
===================================================================
--- branches/release-0.93/docs/utils/gensites.txt 2011-09-06 19:12:14 UTC (rev 5078)
+++ trunk/docs/utils/gensites.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -1,235 +0,0 @@
-Gensites
-==========
-:toc:
-:icons:
-:website: http://www.ci.uchicago.edu/swift
-:numbered:
-
-Overview
---------
-To simplify this configuration process, versions of Swift starting with
-0.92 include a utility called gensites. The gensites command is used
-to generate a sites.xml file for running a swift workflow on a given
-site. It accomplishes this by using a series of templates. The templates
-used by gensites are the same templates used for internal testing, so
-they are likely up to date and known to work on a given site.
-
-
-Viewing All Available Site Templates
-------------------------------------
-To view a list of all available templates, run the following command:
-
------
-$ gensites -T
------
-
-You should see output similar to this:
------
-intrepid
-local
-local-pbs-coasters
-pads
-queenbee
-sge-local
-ssh
-ssh-pbs-coasters
-surveyor
------
-
-You will notice that the templates can be specific to a particular set
-of machines like Intrepid and Queenbee, or they may be more general and
-aim to work across a variety of machines, as in the case of
-local-pbs-coasters. Gensites will look in three directories for available
-templates: your current directory, $SWIFT_HOME/etc/sites and $HOME/.swift/sites.
-
-Listing the Template
---------------------
-To view the contents of a template, type:
-
------
-$ gensites -l templatename
------
-
-Running this command will print some information about the template and give
-you an idea of what settings you will need to specify. You should see
-something like this:
-
------
-Description: Template for Intrepid. More information about this system can be found at http://www.alcf.anl.gov/support/gettingstarted/index.php
-Required Tokens: host work project queue
------
-
-The required tokens are required to properly use the templates. These are placeholder
-values you will need to specify in the
-following steps.
-
-Providing Site Specific Values
-------------------------------
-
-The gensites script needs to know how to replace the placeholder values
-in the template. This is done by configuring the swift.properties file.
-Gensites will first look for a swift.properties file in the current
-directory. If it does not exist, it will next look in $HOME/.swift.
-
-To add site specific values to swift.properties, add a line in the
-follow format:
-
------
-#site templatename setting=value
------
-
-Continuing the previous example, here is what you could add to
-swift.properties to replace the values of project, queue and work for
-the surveyor template:
-
------
-#site surveyor project=MyProject
-#site surveyor queue=MyQueue
-#site surveyor work=/path/to/workdir
------
-
-Now, running the command 'gensites surveyor' will produce the following
-valid configuration file:
-
------
-<config>
- <pool handle="surveyor">
- <filesystem provider="local" />
- <execution provider="coaster" jobmanager="local:cobalt"/>
- <profile namespace="globus" key="project">MyProject</profile>
- <profile namespace="globus" key="queue">MyQueue</profile>
- <profile namespace="globus" key="kernelprofile">zeptoos</profile>
- <profile namespace="globus" key="alcfbgpnat">true</profile>
- <profile namespace="karajan" key="jobthrottle">21</profile>
- <profile namespace="karajan" key="initialScore">10000</profile>
- <profile namespace="globus" key="workersPerNode">1</profile>
- <profile namespace="globus" key="workerLoggingLevel">DEBUG</profile>
- <profile namespace="globus" key="slots">1</profile>
- <profile namespace="globus" key="maxTime">900</profile>
- <profile namespace="globus" key="nodeGranularity">64</profile>
- <profile namespace="globus" key="maxNodes">64</profile>
- <workdirectory>/path/to/workdir</workdirectory>
- </pool>
-</config>
------
-
-Generating Application Configurations with Gensites
----------------------------------------------------
-Gensites can also be used to create a valid application catalog, commonly
-called tc.data. Here are some examples of how to specify applications within
-your swift.properties file:
-
------
-#app intrepid echo=/usr/bin/echo
------
-
-This first example shows a site specific application. The #app definition
-tells gensites this is related to an application rather than a #site
-definition. In the second part, echo=/usr/bin/echo, the left hand side
-is the name of the application that will be called from within swift. The
-right hand site is the path name which points to the binary.
-
------
-#app intrepid echo=$HOME/bin/echo
------
-
-Environment variables will be interpreted and converted to full path names
-for Swift.
-
------
-#app intrepid echo=bin/echo
------
-
-Gensites can take relative paths (relative to your current directory) and
-translate them to full path names for Swift.
-
-WARNING: Running gensites with #app definitions will replace any file
-called tc.data in your current directory. If a file called tc.data exists,
-it will be renamed to tc.data.old. If you run gensites twice, the original
-contents of your tc.data will be lost. Please either rename your tc file
-or copy to a different location.
-
-Running Swift With the New Configuration
-----------------------------------------
-
-Now that the gensites is configured and producing a valid configuration
-file, Swift needs to know to use it. The first step is to create a
-unique config file based on the preferences you specified.
-
------
-$ gensites surveyor > myconfig.xml
------
-
-This will send the output of gensites to myconfig.xml. This example will
-use a swift.properties location in the default directories (your current
-directory, ~/.swift/swift.properties). To specify a different location
-to the swift.properties, use:
-
------
-$ gensites surveyor -p myswift.properties > myconfig.xml
------
-
-Next, provide the configuration filename to swift:
------
-$ swift -sites.file myconfig.xml myscript.swift
------
-
-Alternatively, if you have specified applications, be sure to load that into
-Swift
------
-$ swift -sites.file myconfig.xml -tc.file tc.data mycript.swift
------
-
-Providing Default Values for All Templates
-------------------------------------------
-It is also possible to specify a default value for a setting, regardless
-of template you use. If you want to set your queue to default to "fast"
-across all templates, you can do this by omitting the template name.
-Consider the following swift.properties:
-
------
-#site queue=fast
-#site surveyor project=MyProject
-#site surveyor work=/path/to/workdir
------
-By omitting the template name, the default value for queue on surveyor
-(and any other template you use) will be set to "fast". One thing to
-keep in mind when setting default values is that order matters. Be sure
-to set your default values first before setting template specific values.
-
-Just like the #site definitions, when a site name is not specified, an app
-will be created for every site that is defined in your template.
------
-#app echo=/bin/echo
------
-
-More Help
----------
-The gensites script provides additional options not discussed here, such
-as using templates and swift.properties in non-standard directories. For
-more information, run gensites -h. Here is a full list of all options
-available.
-
------
-$ gensites -help
-
- usage: gensites template [-p properties.file] [-L template_directory] [-h] [-T] [-l]
-
- template Name of template to use
- -p properties.file Specify a swift.properties to use
- -L template_directory Specify a non-standard template directory
- -T List all templates available
- -h Help / usage information
- -l List the contents of a specific template
-
- Examples:
-
- Create a site configuration file for sites.xml using default properties.file in current directory
- $ gensites pads > sites.xml
-
- Use a specific properties file for a site
- $ gensites -p sites.properties pads > sites.xml
-
- Specify a non-standard directory where templates are located
- $ gensites -L template.dir pads > sites.xml
------
Copied: trunk/docs/utils/gensites.txt (from rev 5078, branches/release-0.93/docs/utils/gensites.txt)
===================================================================
--- trunk/docs/utils/gensites.txt (rev 0)
+++ trunk/docs/utils/gensites.txt 2011-09-07 16:32:23 UTC (rev 5079)
@@ -0,0 +1,235 @@
+Gensites
+==========
+:toc:
+:icons:
+:website: http://www.ci.uchicago.edu/swift
+:numbered:
+
+Overview
+--------
+To simplify this configuration process, versions of Swift starting with
+0.92 include a utility called gensites. The gensites command is used
+to generate a sites.xml file for running a swift workflow on a given
+site. It accomplishes this by using a series of templates. The templates
+used by gensites are the same templates used for internal testing, so
+they are likely up to date and known to work on a given site.
+
+
+Viewing All Available Site Templates
+------------------------------------
+To view a list of all available templates, run the following command:
+
+-----
+$ gensites -T
+-----
+
+You should see output similar to this:
+-----
+intrepid
+local
+local-pbs-coasters
+pads
+queenbee
+sge-local
+ssh
+ssh-pbs-coasters
+surveyor
+-----
+
+You will notice that the templates can be specific to a particular set
+of machines like Intrepid and Queenbee, or they may be more general and
+aim to work across a variety of machines, as in the case of
+local-pbs-coasters. Gensites will look in three directories for available
+templates: your current directory, $SWIFT_HOME/etc/sites and $HOME/.swift/sites.
+
+Listing the Template
+--------------------
+To view the contents of a template, type:
+
+-----
+$ gensites -l templatename
+-----
+
+Running this command will print some information about the template and give
+you an idea of what settings you will need to specify. You should see
+something like this:
+
+-----
+Description: Template for Intrepid. More information about this system can be found at http://www.alcf.anl.gov/support/gettingstarted/index.php
+Required Tokens: host work project queue
+-----
+
+The required tokens are required to properly use the templates. These are placeholder
+values you will need to specify in the
+following steps.
+
+Providing Site Specific Values
+------------------------------
+
+The gensites script needs to know how to replace the placeholder values
+in the template. This is done by configuring the swift.properties file.
+Gensites will first look for a swift.properties file in the current
+directory. If it does not exist, it will next look in $HOME/.swift.
+
+To add site specific values to swift.properties, add a line in the
+follow format:
+
+-----
+#site templatename setting=value
+-----
+
+Continuing the previous example, here is what you could add to
+swift.properties to replace the values of project, queue and work for
+the surveyor template:
+
+-----
+#site surveyor project=MyProject
+#site surveyor queue=MyQueue
+#site surveyor work=/path/to/workdir
+-----
+
+Now, running the command 'gensites surveyor' will produce the following
+valid configuration file:
+
+-----
+<config>
+ <pool handle="surveyor">
+ <filesystem provider="local" />
+ <execution provider="coaster" jobmanager="local:cobalt"/>
+ <profile namespace="globus" key="project">MyProject</profile>
+ <profile namespace="globus" key="queue">MyQueue</profile>
+ <profile namespace="globus" key="kernelprofile">zeptoos</profile>
+ <profile namespace="globus" key="alcfbgpnat">true</profile>
+ <profile namespace="karajan" key="jobthrottle">21</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <profile namespace="globus" key="workersPerNode">1</profile>
+ <profile namespace="globus" key="workerLoggingLevel">DEBUG</profile>
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="maxTime">900</profile>
+ <profile namespace="globus" key="nodeGranularity">64</profile>
+ <profile namespace="globus" key="maxNodes">64</profile>
+ <workdirectory>/path/to/workdir</workdirectory>
+ </pool>
+</config>
+-----
+
+Generating Application Configurations with Gensites
+---------------------------------------------------
+Gensites can also be used to create a valid application catalog, commonly
+called tc.data. Here are some examples of how to specify applications within
+your swift.properties file:
+
+-----
+#app intrepid echo=/usr/bin/echo
+-----
+
+This first example shows a site specific application. The #app definition
+tells gensites this is related to an application rather than a #site
+definition. In the second part, echo=/usr/bin/echo, the left hand side
+is the name of the application that will be called from within swift. The
+right hand site is the path name which points to the binary.
+
+-----
+#app intrepid echo=$HOME/bin/echo
+-----
+
+Environment variables will be interpreted and converted to full path names
+for Swift.
+
+-----
+#app intrepid echo=bin/echo
+-----
+
+Gensites can take relative paths (relative to your current directory) and
+translate them to full path names for Swift.
+
+WARNING: Running gensites with #app definitions will replace any file
+called tc.data in your current directory. If a file called tc.data exists,
+it will be renamed to tc.data.old. If you run gensites twice, the original
+contents of your tc.data will be lost. Please either rename your tc file
+or copy to a different location.
+
+Running Swift With the New Configuration
+----------------------------------------
+
+Now that the gensites is configured and producing a valid configuration
+file, Swift needs to know to use it. The first step is to create a
+unique config file based on the preferences you specified.
+
+-----
+$ gensites surveyor > myconfig.xml
+-----
+
+This will send the output of gensites to myconfig.xml. This example will
+use a swift.properties location in the default directories (your current
+directory, ~/.swift/swift.properties). To specify a different location
+to the swift.properties, use:
+
+-----
+$ gensites surveyor -p myswift.properties > myconfig.xml
+-----
+
+Next, provide the configuration filename to swift:
+-----
+$ swift -sites.file myconfig.xml myscript.swift
+-----
+
+Alternatively, if you have specified applications, be sure to load that into
+Swift
+-----
+$ swift -sites.file myconfig.xml -tc.file tc.data mycript.swift
+-----
+
+Providing Default Values for All Templates
+------------------------------------------
+It is also possible to specify a default value for a setting, regardless
+of template you use. If you want to set your queue to default to "fast"
+across all templates, you can do this by omitting the template name.
+Consider the following swift.properties:
+
+-----
+#site queue=fast
+#site surveyor project=MyProject
+#site surveyor work=/path/to/workdir
+-----
+By omitting the template name, the default value for queue on surveyor
+(and any other template you use) will be set to "fast". One thing to
+keep in mind when setting default values is that order matters. Be sure
+to set your default values first before setting template specific values.
+
+Just like the #site definitions, when a site name is not specified, an app
+will be created for every site that is defined in your template.
+-----
+#app echo=/bin/echo
+-----
+
+More Help
+---------
+The gensites script provides additional options not discussed here, such
+as using templates and swift.properties in non-standard directories. For
+more information, run gensites -h. Here is a full list of all options
+available.
+
+-----
+$ gensites -help
+
+ usage: gensites template [-p properties.file] [-L template_directory] [-h] [-T] [-l]
+
+ template Name of template to use
+ -p properties.file Specify a swift.properties to use
+ -L template_directory Specify a non-standard template directory
+ -T List all templates available
+ -h Help / usage information
+ -l List the contents of a specific template
+
+ Examples:
+
+ Create a site configuration file for sites.xml using default properties.file in current directory
+ $ gensites pads > sites.xml
+
+ Use a specific properties file for a site
+ $ gensites -p sites.properties pads > sites.xml
+
+ Specify a non-standard directory where templates are located
+ $ gensites -L template.dir pads > sites.xml
+-----
More information about the Swift-commit
mailing list