[Swift-commit] r4967 - trunk/docs/siteguide
davidk at ci.uchicago.edu
davidk at ci.uchicago.edu
Tue Aug 9 06:16:34 CDT 2011
Author: davidk
Date: 2011-08-09 06:16:34 -0500 (Tue, 09 Aug 2011)
New Revision: 4967
Removed:
trunk/docs/siteguide/siteguide.html
Modified:
trunk/docs/siteguide/pads
Log:
Fixed broken path
Modified: trunk/docs/siteguide/pads
===================================================================
--- trunk/docs/siteguide/pads 2011-08-09 09:33:48 UTC (rev 4966)
+++ trunk/docs/siteguide/pads 2011-08-09 11:16:34 UTC (rev 4967)
@@ -165,7 +165,7 @@
The swift script we will run is called catsn.swift. It simply cats a file and saves the result. This is a nice simple test to ensure jobs are running correctly. Create a file called data.txt which contains some simple input - a "hello world" will do the trick.
-----
-include::../../examples/catsn.swift[]
+include::../../examples/misc/catsn.swift[]
-----
Running Swift
Deleted: trunk/docs/siteguide/siteguide.html
===================================================================
--- trunk/docs/siteguide/siteguide.html 2011-08-09 09:33:48 UTC (rev 4966)
+++ trunk/docs/siteguide/siteguide.html 2011-08-09 11:16:34 UTC (rev 4967)
@@ -1,1160 +0,0 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
- "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
-<head>
-<meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
-<meta name="generator" content="AsciiDoc 8.6.4" />
-<title>Site Configuration Guide</title>
-<style type="text/css">
-/* Sans-serif font. */
-h1, h2, h3, h4, h5, h6,
-div.title, caption.title,
-thead, p.table.header,
-div#toctitle,
-span#author, span#revnumber, span#revdate, span#revremark,
-div#footer {
- font-family: Arial,Helvetica,sans-serif;
-}
-
-/* Serif font. */
-div.sectionbody {
- font-family: Georgia,"Times New Roman",Times,serif;
-}
-
-/* Monospace font. */
-tt {
- font-size: inherit;
-}
-
-body {
- margin: 1em 5% 1em 5%;
-}
-
-a {
- color: blue;
- text-decoration: underline;
-}
-a:visited {
- color: fuchsia;
-}
-
-em {
- font-style: italic;
- color: navy;
-}
-
-strong {
- font-weight: bold;
- color: #083194;
-}
-
-tt {
- font-size: inherit;
- color: navy;
-}
-
-h1, h2, h3, h4, h5, h6 {
- color: #527bbd;
- margin-top: 1.2em;
- margin-bottom: 0.5em;
- line-height: 1.3;
-}
-
-h1, h2, h3 {
- border-bottom: 2px solid silver;
-}
-h2 {
- padding-top: 0.5em;
-}
-h3 {
- float: left;
-}
-h3 + * {
- clear: left;
-}
-
-div.sectionbody {
- margin-left: 0;
-}
-
-hr {
- border: 1px solid silver;
-}
-
-p {
- margin-top: 0.5em;
- margin-bottom: 0.5em;
-}
-
-ul, ol, li > p {
- margin-top: 0;
-}
-ul > li { color: #aaa; }
-ul > li > * { color: black; }
-
-pre {
- padding: 0;
- margin: 0;
-}
-
-span#author {
- color: #527bbd;
- font-weight: bold;
- font-size: 1.1em;
-}
-span#email {
-}
-span#revnumber, span#revdate, span#revremark {
-}
-
-div#footer {
- font-size: small;
- border-top: 2px solid silver;
- padding-top: 0.5em;
- margin-top: 4.0em;
-}
-div#footer-text {
- float: left;
- padding-bottom: 0.5em;
-}
-div#footer-badges {
- float: right;
- padding-bottom: 0.5em;
-}
-
-div#preamble {
- margin-top: 1.5em;
- margin-bottom: 1.5em;
-}
-div.tableblock, div.imageblock, div.exampleblock, div.verseblock,
-div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
-div.admonitionblock {
- margin-top: 1.0em;
- margin-bottom: 1.5em;
-}
-div.admonitionblock {
- margin-top: 2.0em;
- margin-bottom: 2.0em;
- margin-right: 10%;
- color: #606060;
-}
-
-div.content { /* Block element content. */
- padding: 0;
-}
-
-/* Block element titles. */
-div.title, caption.title {
- color: #527bbd;
- font-weight: bold;
- text-align: left;
- margin-top: 1.0em;
- margin-bottom: 0.5em;
-}
-div.title + * {
- margin-top: 0;
-}
-
-td div.title:first-child {
- margin-top: 0.0em;
-}
-div.content div.title:first-child {
- margin-top: 0.0em;
-}
-div.content + div.title {
- margin-top: 0.0em;
-}
-
-div.sidebarblock > div.content {
- background: #ffffee;
- border: 1px solid #dddddd;
- border-left: 4px solid #f0f0f0;
- padding: 0.5em;
-}
-
-div.listingblock > div.content {
- border: 1px solid #dddddd;
- border-left: 5px solid #f0f0f0;
- background: #f8f8f8;
- padding: 0.5em;
-}
-
-div.quoteblock, div.verseblock {
- padding-left: 1.0em;
- margin-left: 1.0em;
- margin-right: 10%;
- border-left: 5px solid #f0f0f0;
- color: #777777;
-}
-
-div.quoteblock > div.attribution {
- padding-top: 0.5em;
- text-align: right;
-}
-
-div.verseblock > pre.content {
- font-family: inherit;
- font-size: inherit;
-}
-div.verseblock > div.attribution {
- padding-top: 0.75em;
- text-align: left;
-}
-/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
-div.verseblock + div.attribution {
- text-align: left;
-}
-
-div.admonitionblock .icon {
- vertical-align: top;
- font-size: 1.1em;
- font-weight: bold;
- text-decoration: underline;
- color: #527bbd;
- padding-right: 0.5em;
-}
-div.admonitionblock td.content {
- padding-left: 0.5em;
- border-left: 3px solid #dddddd;
-}
-
-div.exampleblock > div.content {
- border-left: 3px solid #dddddd;
- padding-left: 0.5em;
-}
-
-div.imageblock div.content { padding-left: 0; }
-span.image img { border-style: none; }
-a.image:visited { color: white; }
-
-dl {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-dt {
- margin-top: 0.5em;
- margin-bottom: 0;
- font-style: normal;
- color: navy;
-}
-dd > *:first-child {
- margin-top: 0.1em;
-}
-
-ul, ol {
- list-style-position: outside;
-}
-ol.arabic {
- list-style-type: decimal;
-}
-ol.loweralpha {
- list-style-type: lower-alpha;
-}
-ol.upperalpha {
- list-style-type: upper-alpha;
-}
-ol.lowerroman {
- list-style-type: lower-roman;
-}
-ol.upperroman {
- list-style-type: upper-roman;
-}
-
-div.compact ul, div.compact ol,
-div.compact p, div.compact p,
-div.compact div, div.compact div {
- margin-top: 0.1em;
- margin-bottom: 0.1em;
-}
-
-div.tableblock > table {
- border: 3px solid #527bbd;
-}
-thead, p.table.header {
- font-weight: bold;
- color: #527bbd;
-}
-tfoot {
- font-weight: bold;
-}
-td > div.verse {
- white-space: pre;
-}
-p.table {
- margin-top: 0;
-}
-/* Because the table frame attribute is overriden by CSS in most browsers. */
-div.tableblock > table[frame="void"] {
- border-style: none;
-}
-div.tableblock > table[frame="hsides"] {
- border-left-style: none;
- border-right-style: none;
-}
-div.tableblock > table[frame="vsides"] {
- border-top-style: none;
- border-bottom-style: none;
-}
-
-
-div.hdlist {
- margin-top: 0.8em;
- margin-bottom: 0.8em;
-}
-div.hdlist tr {
- padding-bottom: 15px;
-}
-dt.hdlist1.strong, td.hdlist1.strong {
- font-weight: bold;
-}
-td.hdlist1 {
- vertical-align: top;
- font-style: normal;
- padding-right: 0.8em;
- color: navy;
-}
-td.hdlist2 {
- vertical-align: top;
-}
-div.hdlist.compact tr {
- margin: 0;
- padding-bottom: 0;
-}
-
-.comment {
- background: yellow;
-}
-
-.footnote, .footnoteref {
- font-size: 0.8em;
-}
-
-span.footnote, span.footnoteref {
- vertical-align: super;
-}
-
-#footnotes {
- margin: 20px 0 20px 0;
- padding: 7px 0 0 0;
-}
-
-#footnotes div.footnote {
- margin: 0 0 5px 0;
-}
-
-#footnotes hr {
- border: none;
- border-top: 1px solid silver;
- height: 1px;
- text-align: left;
- margin-left: 0;
- width: 20%;
- min-width: 100px;
-}
-
-div.colist td {
- padding-right: 0.5em;
- padding-bottom: 0.3em;
- vertical-align: top;
-}
-div.colist td img {
- margin-top: 0.3em;
-}
-
- at media print {
- div#footer-badges { display: none; }
-}
-
-div#toc {
- margin-bottom: 2.5em;
-}
-
-div#toctitle {
- color: #527bbd;
- font-size: 1.1em;
- font-weight: bold;
- margin-top: 1.0em;
- margin-bottom: 0.1em;
-}
-
-div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
- margin-top: 0;
- margin-bottom: 0;
-}
-div.toclevel2 {
- margin-left: 2em;
- font-size: 0.9em;
-}
-div.toclevel3 {
- margin-left: 4em;
- font-size: 0.9em;
-}
-div.toclevel4 {
- margin-left: 6em;
- font-size: 0.9em;
-}
-
-span.aqua { color: aqua; }
-span.black { color: black; }
-span.blue { color: blue; }
-span.fuchsia { color: fuchsia; }
-span.gray { color: gray; }
-span.green { color: green; }
-span.lime { color: lime; }
-span.maroon { color: maroon; }
-span.navy { color: navy; }
-span.olive { color: olive; }
-span.purple { color: purple; }
-span.red { color: red; }
-span.silver { color: silver; }
-span.teal { color: teal; }
-span.white { color: white; }
-span.yellow { color: yellow; }
-
-span.aqua-background { background: aqua; }
-span.black-background { background: black; }
-span.blue-background { background: blue; }
-span.fuchsia-background { background: fuchsia; }
-span.gray-background { background: gray; }
-span.green-background { background: green; }
-span.lime-background { background: lime; }
-span.maroon-background { background: maroon; }
-span.navy-background { background: navy; }
-span.olive-background { background: olive; }
-span.purple-background { background: purple; }
-span.red-background { background: red; }
-span.silver-background { background: silver; }
-span.teal-background { background: teal; }
-span.white-background { background: white; }
-span.yellow-background { background: yellow; }
-
-span.big { font-size: 2em; }
-span.small { font-size: 0.6em; }
-</style>
-<script type="text/javascript">
-/*<+'])');
- // Function that scans the DOM tree for header elements (the DOM2
- // nodeIterator API would be a better technique but not supported by all
- // browsers).
- var iterate = function (el) {
- for (var i = el.firstChild; i != null; i = i.nextSibling) {
- if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
- var mo = re.exec(i.tagName);
- if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
- result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
- }
- iterate(i);
- }
- }
- }
- iterate(el);
- return result;
- }
-
- var toc = document.getElementById("toc");
- var entries = tocEntries(document.getElementById("content"), toclevels);
- for (var i = 0; i < entries.length; ++i) {
- var entry = entries[i];
- if (entry.element.id == "")
- entry.element.id = "_toc_" + i;
- var a = document.createElement("a");
- a.href = "#" + entry.element.id;
- a.appendChild(document.createTextNode(entry.text));
- var div = document.createElement("div");
- div.appendChild(a);
- div.className = "toclevel" + entry.toclevel;
- toc.appendChild(div);
- }
- if (entries.length == 0)
- toc.parentNode.removeChild(toc);
-},
-
-
-/////////////////////////////////////////////////////////////////////
-// Footnotes generator
-/////////////////////////////////////////////////////////////////////
-
-/* Based on footnote generation code from:
- * http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
- */
-
-footnotes: function () {
- var cont = document.getElementById("content");
- var noteholder = document.getElementById("footnotes");
- var spans = cont.getElementsByTagName("span");
- var refs = {};
- var n = 0;
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnote") {
- n++;
- // Use [\s\S] in place of . so multi-line matches work.
- // Because JavaScript has no s (dotall) regex flag.
- note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
- noteholder.innerHTML +=
- "<div class='footnote' id='_footnote_" + n + "'>" +
- "<a href='#_footnoteref_" + n + "' title='Return to text'>" +
- n + "</a>. " + note + "</div>";
- spans[i].innerHTML =
- "[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- var id =spans[i].getAttribute("id");
- if (id != null) refs["#"+id] = n;
- }
- }
- if (n == 0)
- noteholder.parentNode.removeChild(noteholder);
- else {
- // Process footnoterefs.
- for (i=0; i<spans.length; i++) {
- if (spans[i].className == "footnoteref") {
- var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
- href = href.match(/#.*/)[0]; // Because IE return full URL.
- n = refs[href];
- spans[i].innerHTML =
- "[<a href='#_footnote_" + n +
- "' title='View footnote' class='footnote'>" + n + "</a>]";
- }
- }
- }
-}
-
-}
-/*]]>*/
-</script>
-</head>
-<body class="article" style="max-width:750px">
-<div id="header">
-<h1>Site Configuration Guide</h1>
-<div id="toc">
- <div id="toctitle">Table of Contents</div>
- <noscript><p><b>JavaScript must be enabled in your browser to display the table of contents.</b></p></noscript>
-</div>
-</div>
-<div id="content">
-<div class="sect1">
-<h2 id="_pads">1. PADS</h2>
-<div class="sectionbody">
-<div class="sect2">
-<h3 id="_requesting_access">1.1. Requesting Access</h3>
-<div class="paragraph"><p>If you do not already have a Computation Institute account, you can request
-one at <a href="https://www.ci.uchicago.edu/accounts/">https://www.ci.uchicago.edu/accounts/</a>. This page will give you a list
-of resources you can request access to. Be sure that PADS is selected. If
-you already have an existing CI account, but do not have access to PADS,
-send an email to <a href="mailto:support at ci.uchicago.edu">support at ci.uchicago.edu</a> to request access.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_ssh_keys">1.2. SSH Keys</h3>
-<div class="paragraph"><p>Before accessing PADS, be sure to have your SSH keys configured correctly.
-There is some basic information about SSH and how to generate your key at
-<a href="http://www.ci.uchicago.edu/wiki/bin/view/Resources/SshKeys">http://www.ci.uchicago.edu/wiki/bin/view/Resources/SshKeys</a>. Once you have
-followed those instructions, you can add your key at
-<a href="https://www.ci.uchicago.edu/support/sshkeys/">https://www.ci.uchicago.edu/support/sshkeys/</a>.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_connecting_to_a_login_node">1.3. Connecting to a login node</h3>
-<div class="paragraph"><p>Once your keys are configured, you should be able to access a PADS login
-node with the following command:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>ssh yourusername at login.pads.ci.uchicago.edu</tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_configuring_softenv">1.4. Configuring softenv</h3>
-<div class="paragraph"><p>Softenv is a system used for managing applications. In order to run Swift,
-the softenv environment will have to be modified slightly. Softenv is
-configured by a file in your home directory called .soft. Edit this file
-to look like this:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>+java-sun
-+maui
-+torque
- at default</tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_downloading_and_building_swift">1.5. Downloading and building Swift</h3>
-<div class="paragraph"><p>The most recent versions of Swift can be found at
-<a href="http://www.ci.uchicago.edu/swift/downloads/index.php">http://www.ci.uchicago.edu/swift/downloads/index.php</a>. Follow the instructions
-provided on that site to download and build Swift.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_adding_swift_to_your_path">1.6. Adding Swift to your PATH</h3>
-<div class="paragraph"><p>Once you have installed Swift, add the Swift binary to your PATH so you can
-easily run it from any directory.</p></div>
-<div class="paragraph"><p>In your home directory, edit the file ".bashrc".</p></div>
-<div class="paragraph"><p>If you have installed Swift via a source repository, add the following line
-at the bottom of .bashrc.</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>export PATH=$PATH:$HOME/cog/modules/swift/dist/swift-svn/bin</tt></pre>
-</div></div>
-<div class="paragraph"><p>If you have installed Swift via a binary package, add this line:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>export PATH=$PATH:$HOME/swift-<version>/bin</tt></pre>
-</div></div>
-<div class="paragraph"><p>Replace <version> with the actual name of the swift directory in the example above.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_what_you_need_to_know_before_running_swift">1.7. What You Need To Know Before Running Swift</h3>
-<div class="paragraph"><p>Before you can create a Swift configuration file, there are some things
-you will need to know.</p></div>
-<div class="sect3">
-<h4 id="_swift_work_directory">1.7.1. Swift Work Directory</h4>
-<div class="paragraph"><p>The Swift work directory is a directory which Swift uses for processing work.
-This directory needs to be writable. Common options for this are:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>/home/username/swiftwork
-/home/username/work
-/tmp</tt></pre>
-</div></div>
-</div>
-<div class="sect3">
-<h4 id="_which_project_s_are_you_a_member_of">1.7.2. Which project(s) are you a member of?</h4>
-<div class="paragraph"><p>PADS requires that you are a member of a project. You can determine this by
-running the following command:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ projects --available
-
-The following projects are available for your use
-
-Project PI Title
-
-CI-CCR000013 Michael Wilde The Swift Parallel Scripting System</tt></pre>
-</div></div>
-<div class="paragraph"><p>If you are not a member of a project, you must first request access
-to a project at <a href="http://www.ci.uchicago.edu/hpc/projects">http://www.ci.uchicago.edu/hpc/projects</a>.</p></div>
-<div class="paragraph"><p>You should make sure that you have a project set as default. Run
-the projects command with no arguments to determine if you have a default.</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ projects
-You have no default project set.</tt></pre>
-</div></div>
-<div class="paragraph"><p>To set your default project, use projects --set</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ projects --set CI-CCR000013 --all
-Your default project for all CI clusters has been set to CI-CCR000013.</tt></pre>
-</div></div>
-</div>
-<div class="sect3">
-<h4 id="_determine_your_queue">1.7.3. Determine your Queue</h4>
-<div class="paragraph"><p>PADS has several different queues you can submit jobs to depending on
-the type of work you will be doing. The command "qstat -q" will print
-the most up to date list of this information.</p></div>
-<div class="tableblock">
-<table rules="all"
-width="100%"
-frame="border"
-cellspacing="0" cellpadding="4">
-<caption class="title">Table 1. PADS Queues</caption>
-<col width="11%" />
-<col width="11%" />
-<col width="11%" />
-<col width="11%" />
-<col width="11%" />
-<col width="11%" />
-<col width="11%" />
-<col width="11%" />
-<col width="11%" />
-<thead>
-<tr>
-<th align="left" valign="top">Queue </th>
-<th align="left" valign="top">Memory</th>
-<th align="left" valign="top">CPU Time</th>
-<th align="left" valign="top">Walltime</th>
-<th align="left" valign="top">Node</th>
-<th align="left" valign="top">Run</th>
-<th align="left" valign="top">Que</th>
-<th align="left" valign="top">Lm </th>
-<th align="left" valign="top">State</th>
-</tr>
-</thead>
-<tbody>
-<tr>
-<td align="left" valign="top"><p class="table">route</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">0</p></td>
-<td align="left" valign="top"><p class="table">0</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">E R</p></td>
-</tr>
-<tr>
-<td align="left" valign="top"><p class="table">short</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">04:00:00</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">64</p></td>
-<td align="left" valign="top"><p class="table">0</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">E R</p></td>
-</tr>
-<tr>
-<td align="left" valign="top"><p class="table">extended</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">0</p></td>
-<td align="left" valign="top"><p class="table">0</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">E R</p></td>
-</tr>
-<tr>
-<td align="left" valign="top"><p class="table">fast</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">01:00:00</p></td>
-<td align="left" valign="top"><p class="table">1</p></td>
-<td align="left" valign="top"><p class="table">0</p></td>
-<td align="left" valign="top"><p class="table">152</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">E R</p></td>
-</tr>
-<tr>
-<td align="left" valign="top"><p class="table">long</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">24:00:00</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">232</p></td>
-<td align="left" valign="top"><p class="table">130</p></td>
-<td align="left" valign="top"><p class="table">— </p></td>
-<td align="left" valign="top"><p class="table">E R</p></td>
-</tr>
-</tbody>
-</table>
-</div>
-</div>
-</div>
-<div class="sect2">
-<h3 id="_generating_configuration_files">1.8. Generating Configuration Files</h3>
-<div class="paragraph"><p>Now that you know what queue to use, your project, and your work directory, it is time to
-set up Swift. Swift uses a configuration file called sites.xml to determine how it should run.
-There are two methods you can use for creating this file. You can manually edit
-the configuration file, or generate it with a utility called gensites.</p></div>
-<div class="sect3">
-<h4 id="_manually_editing_sites_xml">1.8.1. Manually Editing sites.xml</h4>
-<div class="paragraph"><p>Below is the template that is used by Swift’s test suite for running on PADS.</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt><config>
-<pool handle="pads-pbs">
- <execution jobmanager="local:pbs" provider="local" url="none" />
- <filesystem provider="local" url="none" />
- <profile key="jobThrottle" namespace="karajan">0</profile>
- <profile namespace="globus" key="project">_QUEUE_</profile>
- <workdirectory>_WORK_</workdirectory>
-</pool>
-</config></tt></pre>
-</div></div>
-<div class="paragraph"><p>The values to note here are the ones that are listed between underscores. In the example above, they are <em>QUEUE</em>, and <em>WORK</em>. Queue is the PADS queue to use and WORK is the swift work directory. These are placeholder values you will need to modify to fit your needs. Copy and paste this template, replace the values, and call it sites.xml.</p></div>
-</div>
-</div>
-<div class="sect2">
-<h3 id="_manually_editing_tc_data">1.9. Manually Editing tc.data</h3>
-<div class="paragraph"><p>Below is the tc.data file used by Swift’s test suite for running on PADS.</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>pads-pbs echo /bin/echo INSTALLED INTEL32::LINUX
-pads-pbs cat /bin/cat INSTALLED INTEL32::LINUX
-pads-pbs ls /bin/ls INSTALLED INTEL32::LINUX
-pads-pbs grep /bin/grep INSTALLED INTEL32::LINUX
-pads-pbs sort /bin/sort INSTALLED INTEL32::LINUX
-pads-pbs paste /bin/paste INSTALLED INTEL32::LINUX
-pads-pbs wc /usr/bin/wc INSTALLED INTEL32::LINUX</tt></pre>
-</div></div>
-<div class="paragraph"><p>Copy these commands and save it as tc.data.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_catsn_swift">1.10. Catsn.swift</h3>
-<div class="paragraph"><p>The swift script we will run is called catsn.swift. It simply cats a file and saves the result. This is a nice simple test to ensure jobs are running correctly. Create a file called data.txt which contains some simple input - a "hello world" will do the trick.</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt></tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_running_swift">1.11. Running Swift</h3>
-<div class="paragraph"><p>Now that everything is in place, run Swift with the following command:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>swift -sites.file sites.xml -tc.file tc.data catsn.swift -n=10</tt></pre>
-</div></div>
-<div class="paragraph"><p>You should see several new files being created, called catsn.0001.out, catsn.0002.out, etc. Each of these
-files should contain the contents of what you placed into data.txt. If this happens, your job has run
-successfully on PADS!</p></div>
-</div>
-<div class="sect2">
-<h3 id="_more_help">1.12. More Help</h3>
-<div class="paragraph"><p>The best place for additional help is the Swift user mailing list. You can subscribe to this list at
-<a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a>. When submitting information, please send your sites.xml file, your tc.data, and any Swift log files that were created during your attempt.</p></div>
-</div>
-</div>
-</div>
-<div class="sect1">
-<h2 id="_beagle">2. Beagle</h2>
-<div class="sectionbody">
-<div class="sect2">
-<h3 id="_requesting_access_2">2.1. Requesting Access</h3>
-<div class="paragraph"><p>If you do not already have a Computation Institute account, you can request
-one at <a href="https://www.ci.uchicago.edu/accounts/">https://www.ci.uchicago.edu/accounts/</a>. This page will give you a list
-of resources you can request access to.
-You already have an existing CI account, but do not have access to Beagle,
-send an email to <a href="mailto:support at ci.uchicago.edu">support at ci.uchicago.edu</a> to request access.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_connecting_to_a_login_node_2">2.2. Connecting to a login node</h3>
-<div class="paragraph"><p>Once you have account, you should be able to access a Beagle login
-node with the following command:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>ssh yourusername at login.beagle.ci.uchicago.edu</tt></pre>
-</div></div>
-<div class="paragraph"><p>Follow the steps outlined below to get started with Swift on Beagle:</p></div>
-<div class="paragraph"><p><strong>step 1.</strong> Load the Swift module on Beagle as follows: <tt>module load swift</tt></p></div>
-<div class="paragraph"><p><strong>step 2.</strong> Create and change to a directory where your Swift related work will
-stay. (say, <tt>mkdir swift-lab</tt>, followed by, <tt>cd swift-lab</tt>)</p></div>
-<div class="paragraph"><p><strong>step 3.</strong> To get started with a simple example running <tt>/bin/cat</tt> to read an
-input file <tt>data.txt</tt> and write to an output file <tt>f.nnn.out</tt>, copy the folder
-at <tt>/home/ketan/catsn</tt> to the above directory. (<tt>cp -r /home/ketan/catsn
-.</tt> followed by <tt>cd catsn</tt>).</p></div>
-<div class="paragraph"><p><strong>step 4.</strong> In the sites file: <tt>beagle-coaster.xml</tt>, make the following two
-changes: <strong>1)</strong> change the path of <tt>workdirectory</tt> to your preferred location
-(say to <tt>/lustre/beagle/$USER/swift-lab/swift.workdir</tt>) and <strong>2)</strong> Change the
-project name to your project (<tt>CI-CCR000013</tt>) . The workdirectory will contain
-execution data related to each run, e.g. wrapper scripts, system information,
-inputs and outputs.</p></div>
-<div class="paragraph"><p><strong>step 5.</strong> Run the example using following commandline (also found in run.sh):
-<tt>swift -config cf -tc.file tc -sites.file beagle-coaster.xml catsn.swift -n=1</tt>
-. You can further change the value of <tt>-n</tt> to any arbitrary number to run that
-many number of concurrent <tt>cat</tt></p></div>
-<div class="paragraph"><p><strong>step 6.</strong> Check the output in the generated <tt>outdir</tt> directory (<tt>ls outdir</tt>)</p></div>
-<div class="paragraph"><p>Note: Running from sandbox node or requesting 1 hour walltime for upto 3 nodes
-will get fast prioritized execution. Good for small tests.</p></div>
-</div>
-</div>
-</div>
-<div class="sect1">
-<h2 id="_futuregrid">3. FutureGrid</h2>
-<div class="sectionbody">
-<div class="sect2">
-<h3 id="_downloading_and_building_swift_2">3.1. Downloading and Building Swift</h3>
-<div class="paragraph"><p>The most recent versions of Swift can be found at
-<a href="http://www.ci.uchicago.edu/swift/downloads/index.php">http://www.ci.uchicago.edu/swift/downloads/index.php</a>. To run on futuregrid,
-you will need to download the development/trunk version from SVN.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_adding_swift_to_your_path_2">3.2. Adding Swift to your PATH</h3>
-<div class="paragraph"><p>Once you have installed Swift, add the Swift binary to your PATH so you can
-easily run it from any directory.</p></div>
-<div class="paragraph"><p>In your home directory, edit the file ".bashrc".</p></div>
-<div class="paragraph"><p>If you have installed Swift via a source repository, add the following line
-at the bottom of .bashrc.</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>export PATH=$PATH:$HOME/cog/modules/swift/dist/swift-svn/bin</tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_requesting_futuregrid_access">3.3. Requesting Futuregrid Access</h3>
-<div class="paragraph"><p>If you do not already have a futuregrid account, you can follow the
-instructions at <a href="https://portal.futuregrid.org/gettingstarted">https://portal.futuregrid.org/gettingstarted</a> to get started.
-This page provides information on how to create an account, how to join
-a project, how to set up your SSH keys, and how to create a new project.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_downloading_swift_vm_tools">3.4. Downloading Swift VM Tools</h3>
-<div class="paragraph"><p>A set of scripts based around cloudinitd are used to easily start virtual
-machines. To download, change to your home directory and run the
-following command:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ svn co https://svn.ci.uchicago.edu/svn/vdl2/usertools/swift-vm-boot</tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_download_your_credentials">3.5. Download your Credentials</h3>
-<div class="paragraph"><p>Run the following commands to retrieve your credentials:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ scp yourusername at hotel.futuregrid.org:nimbus_creds.tar.gz .
-$ tar xvfz nimbus_creds.tar.gz</tt></pre>
-</div></div>
-<div class="paragraph"><p>When you extract your credential file, look at the file called
-hotel.conf. Near the bottom of this file will be two settings
-called vws.repository.s3id and vws.repository.s3key. Copy these
-values for the next step.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_configuring_swift_vm_tools">3.6. Configuring Swift VM Tools</h3>
-<div class="paragraph"><p>Change directories into the swift-vm-boot directory. Edit the file
-called env.sh. There are two settings here that you will need to
-modify: FUTUREGRID_IAAS_ACCESS_KEY and FUTUREGRID_IAAS_SECRET_KEY.
-Paste your access key (s3id) and secret key (s3key) here and save
-the file. By default, env.sh requests 2 nodes on hotel and 2 nodes
-on sierra. Change these values as needed. Run install.sh from the
-swift-vm-boot directory to complete the installation of Swift VM
-Tools. The Swift VM Tools may require a fairly recent version of
-Python. If you run into problems while running the install.sh
-script, please try a more recent version of Python and associated
-libraries.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_configuring_coaster_service_conf">3.7. Configuring coaster-service.conf</h3>
-<div class="paragraph"><p>Within your swift distribution’s etc directory is a file called
-coaster-service.conf. There are many options here you can modify
-as needed, but these are the settings that will be required
-to run on futuregrid:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>export WORKER_WORK=/tmp
-export WORKER_MODE=futuregrid
-export SHARED_FILESYSTEM=no
-export WORKER_USERNAME=root
-export SSH_TUNNELING=yes
-export SWIFTVMBOOT_DIR=$HOME/swift-vm-boot
-export WORK=/tmp</tt></pre>
-</div></div>
-<div class="paragraph"><p>These is a brief description of these settings in the coaster-service.conf file.
-Tunneling should be used when you are restricted by a firewall. If your local
-machine has multiple network interfaces, you should also manually set the IP
-address of your machine with export IPADDR=your.ip.address.here.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_starting_the_coaster_service_script">3.8. Starting the Coaster Service Script</h3>
-<div class="paragraph"><p>Now that everything is configured, change to a temporary directory
-and run this command to start the coaster service:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ start-coaster-service</tt></pre>
-</div></div>
-<div class="paragraph"><p>This command will start the VMs, start the required processes on the worker nodes,
-and generate Swift configuration files for you to use. The configuration files
-will be generated in your current directory. These files are sites.xml, tc.data,
-and cf.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_running_swift_2">3.9. Running Swift</h3>
-<div class="paragraph"><p>Now that you have all of your configuration files generated, run the following command:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ swift -sites.file sites.xml -tc.file tc.data -config cf <yourscript.swift></tt></pre>
-</div></div>
-<div class="paragraph"><p>If you would like to create a custom tc file for repeated use, rename it to something other
-than tc.data to prevent it from being overwritten. The sites.xml however will need to be
-regenerated every time you start the coaster service. If you need to repeatedly modify some
-sites.xml options, you may edit the template in Swift’s etc/sites/persistent-coasters. You
-may also create your own custom tc files with the hostname of persistent-coasters. More
-information about this can be found in the Swift userguide at
-<a href="http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html">http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html</a>.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_stopping_the_coaster_service_script">3.10. Stopping the Coaster Service Script</h3>
-<div class="paragraph"><p>To stop the coaster service, run the following command:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ stop-coaster-service</tt></pre>
-</div></div>
-<div class="paragraph"><p>This will kill the coaster service, kill the worker scripts on remote systems and terminate
-the virtual machines that were created during start-coaster-service.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_more_help_2">3.11. More Help</h3>
-<div class="paragraph"><p>The best place for additional help is the Swift user mailing list. You can subscribe to this list at
-<a href="http://mail.ci.uchicago.edu/mailman/listinfo/swift-user">http://mail.ci.uchicago.edu/mailman/listinfo/swift-user</a>. When submitting information, please send
-your sites.xml file, your tc.data, and any error messages you run into.</p></div>
-</div>
-</div>
-</div>
-<div class="sect1">
-<h2 id="_grids_including_osg_and_teragrid">4. Grids, including OSG and TeraGrid</h2>
-<div class="sectionbody">
-<div class="sect2">
-<h3 id="_overview_of_running_on_grid_sites">4.1. Overview of running on grid sites</h3>
-<div class="ulist"><ul>
-<li>
-<p>
-Get a DOEGrids cert. Then register it in the <strong>OSG Engage VO</strong>, and/or map it using <tt>gx-request</tt> on TeraGrid sites.
-</p>
-</li>
-<li>
-<p>
-Run <tt>GridSetup</tt> to configure Swift to use the grid sites. This tests
-for correct operation and creates a "green list" of good sites.
-</p>
-</li>
-<li>
-<p>
-Prepare an installation package for the programs you want to run on
-grid sites via Swift, and install that package using <tt>foreachsite</tt>.
-</p>
-</li>
-<li>
-<p>
-Run <tt>RunWorkers</tt> to start and maintain a pool of Swift workers on
-each site.
-</p>
-</li>
-<li>
-<p>
-Run Swift scripts that use the grid site resources.
-</p>
-</li>
-</ul></div>
-</div>
-<div class="sect2">
-<h3 id="_requesting_access_3">4.2. Requesting Access</h3>
-<div class="paragraph"><p><strong>For OSG</strong>: Obtain a DOEGrids certificate and register the certificate in
-the OSG "Engage" VO following the procedures at:</p></div>
-<div class="paragraph"><p><a href="https://twiki.grid.iu.edu/bin/view/Engagement/EngageNewUserGuide">https://twiki.grid.iu.edu/bin/view/Engagement/EngageNewUserGuide</a></p></div>
-<div class="paragraph"><p><strong><span class="red">FIXME</span></strong>: access to OSG wiki pages may request the user to present
-a certificate. Is this a problem from users without one? If so, make a
-copy of the page on the Swift web.</p></div>
-<div class="paragraph"><p><strong>For TeraGrid</strong>: Obtain a DOEGrids certifcate using the OSG ENgage
-instructions above. Ask a TeraGrid PI to add you to a TeraGrid
-project. Once you obtain a login and project access (via US Mail), use
-gx-request to add your certificate</p></div>
-<div class="paragraph"><p>To run jobs using the procedures documented here you need to be logged
-in to a "submit host" on which you will run Swift and other
-grid-related utilities. This can be any host with the OSG client stack
-or equivalent tools installed. Such hosts include the OSG Engage
-submit host, and the two Swift lab servers
-{bridled,communicado}.ci.uchicago.edu.</p></div>
-<div class="paragraph"><p>Obtain a login on engage-submit.renci.org following instructions on
-the OSG URL above.</p></div>
-<div class="paragraph"><p>Obtain a CI login with access to the Swift lab servers by requesting
-"OSG Gridlab" access at:</p></div>
-<div class="paragraph"><p><a href="http://accounts.ci.uchicago.edu">http://accounts.ci.uchicago.edu</a></p></div>
-</div>
-<div class="sect2">
-<h3 id="_connecting_to_a_submit_host">4.3. Connecting to a submit host</h3>
-<div class="listingblock">
-<div class="content">
-<pre><tt>ssh yourusername at bridled.ci.uchicago.edu
-ssh yourusername at communicado.ci.uchicago.edu
-ssh yourusername at engage-submit.renci.org</tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_downloading_and_install_swift">4.4. Downloading and install Swift</h3>
-<div class="paragraph"><p>The current version of Swift can be downloaded from
-<a href="http://www.ci.uchicago.edu/swift/downloads/index.php">http://www.ci.uchicago.edu/swift/downloads/index.php</a>.</p></div>
-<div class="paragraph"><p>Fetch and untar the latest release.
-Then add the Swift bin/ directory to your PATH. For example:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>cd $HOME
-wget http://www.ci.uchicago.edu/swift/packages/swift-0.92.1.tar.gz
-tar txf swift-0.92.1.tar.gz
-export PATH=$PATH:$HOME/swift-0.92.1/bin</tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_set_up_osg_environment">4.5. Set up OSG environment</h3>
-<div class="paragraph"><p>Depending on your shell type, run:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>source /opt/osg/setup.sh
-or
-source /opt/osg/setup.csh</tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_create_a_voms_grid_proxy">4.6. Create a VOMS Grid proxy</h3>
-<div class="listingblock">
-<div class="content">
-<pre><tt>$ voms-proxy-init -voms Engage -valid 12:00</tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_generating_configuration_files_2">4.7. Generating Configuration Files</h3>
-<div class="listingblock">
-<div class="content">
-<pre><tt>cd $HOME
-mkdir swiftgrid
-cd swiftgrid
-gen_gridsites
-# Wait a few minutes to a few hours for Swift to validate grid sites
-get_greensites >greensites</tt></pre>
-</div></div>
-</div>
-<div class="sect2">
-<h3 id="_running_swift_3">4.8. Running Swift</h3>
-<div class="paragraph"><p>Now that everything is in place, run Swift with the following command:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>swift -sites.file sites.xml -tc.file tc.data catsn.swift -n=10</tt></pre>
-</div></div>
-<div class="paragraph"><p>You should see several new files being created, called catsn.0001.out, catsn.0002.out, etc. Each of these
-files should contain the contents of what you placed into data.txt. If this happens, your job has run
-successfully on PADS!</p></div>
-</div>
-<div class="sect2">
-<h3 id="_more_help_3">4.9. More Help</h3>
-<div class="paragraph"><p>The best place for additional help is the Swift user mailing list. You can subscribe to this list at
-<a href="http://mail.ci.uchicago.edu/mailman/listinfo/swift-user">http://mail.ci.uchicago.edu/mailman/listinfo/swift-user</a>. When submitting information, please send your sites.xml file, your tc.data, and any Swift log files that were created during your attempt.</p></div>
-</div>
-</div>
-</div>
-</div>
-<div id="footnotes"><hr /></div>
-<div id="footer">
-<div id="footer-text">
-Last updated 2011-08-01 12:53:52 CDT
-</div>
-</div>
-</body>
-</html>
More information about the Swift-commit
mailing list