<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="generator" content="AsciiDoc 8.5.2" />
<title>Futuregrid Quickstart Guide</title>
<style type="text/css">
/* Debug borders */
p, li, dt, dd, div, pre, h1, h2, h3, h4, h5, h6 {
/*
border: 1px solid red;
*/
}
body {
margin: 1em 5% 1em 5%;
}
a {
color: blue;
text-decoration: underline;
}
a:visited {
color: fuchsia;
}
em {
font-style: italic;
color: navy;
}
strong {
font-weight: bold;
color: #083194;
}
tt {
color: navy;
}
h1, h2, h3, h4, h5, h6 {
color: #527bbd;
font-family: sans-serif;
margin-top: 1.2em;
margin-bottom: 0.5em;
line-height: 1.3;
}
h1, h2, h3 {
border-bottom: 2px solid silver;
}
h2 {
padding-top: 0.5em;
}
h3 {
float: left;
}
h3 + * {
clear: left;
}
div.sectionbody {
font-family: serif;
margin-left: 0;
}
hr {
border: 1px solid silver;
}
p {
margin-top: 0.5em;
margin-bottom: 0.5em;
}
ul, ol, li > p {
margin-top: 0;
}
pre {
padding: 0;
margin: 0;
}
span#author {
color: #527bbd;
font-family: sans-serif;
font-weight: bold;
font-size: 1.1em;
}
span#email {
}
span#revnumber, span#revdate, span#revremark {
font-family: sans-serif;
}
div#footer {
font-family: sans-serif;
font-size: small;
border-top: 2px solid silver;
padding-top: 0.5em;
margin-top: 4.0em;
}
div#footer-text {
float: left;
padding-bottom: 0.5em;
}
div#footer-badges {
float: right;
padding-bottom: 0.5em;
}
div#preamble {
margin-top: 1.5em;
margin-bottom: 1.5em;
}
div.tableblock, div.imageblock, div.exampleblock, div.verseblock,
div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
div.admonitionblock {
margin-top: 1.0em;
margin-bottom: 1.5em;
}
div.admonitionblock {
margin-top: 2.0em;
margin-bottom: 2.0em;
margin-right: 10%;
color: #606060;
}
div.content { /* Block element content. */
padding: 0;
}
/* Block element titles. */
div.title, caption.title {
color: #527bbd;
font-family: sans-serif;
font-weight: bold;
text-align: left;
margin-top: 1.0em;
margin-bottom: 0.5em;
}
div.title + * {
margin-top: 0;
}
td div.title:first-child {
margin-top: 0.0em;
}
div.content div.title:first-child {
margin-top: 0.0em;
}
div.content + div.title {
margin-top: 0.0em;
}
div.sidebarblock > div.content {
background: #ffffee;
border: 1px solid silver;
padding: 0.5em;
}
div.listingblock > div.content {
border: 1px solid silver;
background: #f4f4f4;
padding: 0.5em;
}
div.quoteblock, div.verseblock {
padding-left: 1.0em;
margin-left: 1.0em;
margin-right: 10%;
border-left: 5px solid #dddddd;
color: #777777;
}
div.quoteblock > div.attribution {
padding-top: 0.5em;
text-align: right;
}
div.verseblock > div.content {
white-space: pre;
}
div.verseblock > div.attribution {
padding-top: 0.75em;
text-align: left;
}
/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
div.verseblock + div.attribution {
text-align: left;
}
div.admonitionblock .icon {
vertical-align: top;
font-size: 1.1em;
font-weight: bold;
text-decoration: underline;
color: #527bbd;
padding-right: 0.5em;
}
div.admonitionblock td.content {
padding-left: 0.5em;
border-left: 3px solid #dddddd;
}
div.exampleblock > div.content {
border-left: 3px solid #dddddd;
padding-left: 0.5em;
}
div.imageblock div.content { padding-left: 0; }
span.image img { border-style: none; }
a.image:visited { color: white; }
dl {
margin-top: 0.8em;
margin-bottom: 0.8em;
}
dt {
margin-top: 0.5em;
margin-bottom: 0;
font-style: normal;
color: navy;
}
dd > *:first-child {
margin-top: 0.1em;
}
ul, ol {
list-style-position: outside;
}
ol.arabic {
list-style-type: decimal;
}
ol.loweralpha {
list-style-type: lower-alpha;
}
ol.upperalpha {
list-style-type: upper-alpha;
}
ol.lowerroman {
list-style-type: lower-roman;
}
ol.upperroman {
list-style-type: upper-roman;
}
div.compact ul, div.compact ol,
div.compact p, div.compact p,
div.compact div, div.compact div {
margin-top: 0.1em;
margin-bottom: 0.1em;
}
div.tableblock > table {
border: 3px solid #527bbd;
}
thead, p.table.header {
font-family: sans-serif;
font-weight: bold;
}
tfoot {
font-weight: bold;
}
td > div.verse {
white-space: pre;
}
p.table {
margin-top: 0;
}
/* Because the table frame attribute is overriden by CSS in most browsers. */
div.tableblock > table[frame="void"] {
border-style: none;
}
div.tableblock > table[frame="hsides"] {
border-left-style: none;
border-right-style: none;
}
div.tableblock > table[frame="vsides"] {
border-top-style: none;
border-bottom-style: none;
}
div.hdlist {
margin-top: 0.8em;
margin-bottom: 0.8em;
}
div.hdlist tr {
padding-bottom: 15px;
}
dt.hdlist1.strong, td.hdlist1.strong {
font-weight: bold;
}
td.hdlist1 {
vertical-align: top;
font-style: normal;
padding-right: 0.8em;
color: navy;
}
td.hdlist2 {
vertical-align: top;
}
div.hdlist.compact tr {
margin: 0;
padding-bottom: 0;
}
.comment {
background: yellow;
}
.footnote, .footnoteref {
font-size: 0.8em;
}
span.footnote, span.footnoteref {
vertical-align: super;
}
#footnotes {
margin: 20px 0 20px 0;
padding: 7px 0 0 0;
}
#footnotes div.footnote {
margin: 0 0 5px 0;
}
#footnotes hr {
border: none;
border-top: 1px solid silver;
height: 1px;
text-align: left;
margin-left: 0;
width: 20%;
min-width: 100px;
}
@media print {
div#footer-badges { display: none; }
}
div#toc {
margin-bottom: 2.5em;
}
div#toctitle {
color: #527bbd;
font-family: sans-serif;
font-size: 1.1em;
font-weight: bold;
margin-top: 1.0em;
margin-bottom: 0.1em;
}
div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
margin-top: 0;
margin-bottom: 0;
}
div.toclevel2 {
margin-left: 2em;
font-size: 0.9em;
}
div.toclevel3 {
margin-left: 4em;
font-size: 0.9em;
}
div.toclevel4 {
margin-left: 6em;
font-size: 0.9em;
}
/* Workarounds for IE6's broken and incomplete CSS2. */
div.sidebar-content {
background: #ffffee;
border: 1px solid silver;
padding: 0.5em;
}
div.sidebar-title, div.image-title {
color: #527bbd;
font-family: sans-serif;
font-weight: bold;
margin-top: 0.0em;
margin-bottom: 0.5em;
}
div.listingblock div.content {
border: 1px solid silver;
background: #f4f4f4;
padding: 0.5em;
}
div.quoteblock-attribution {
padding-top: 0.5em;
text-align: right;
}
div.verseblock-content {
white-space: pre;
}
div.verseblock-attribution {
padding-top: 0.75em;
text-align: left;
}
div.exampleblock-content {
border-left: 3px solid #dddddd;
padding-left: 0.5em;
}
/* IE6 sets dynamically generated links as visited. */
div#toc a:visited { color: blue; }
</style>
<script type="text/javascript">
/*<+'])');
// Function that scans the DOM tree for header elements (the DOM2
// nodeIterator API would be a better technique but not supported by all
// browsers).
var iterate = function (el) {
for (var i = el.firstChild; i != null; i = i.nextSibling) {
if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
var mo = re.exec(i.tagName);
if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
}
iterate(i);
}
}
}
iterate(el);
return result;
}
var toc = document.getElementById("toc");
var entries = tocEntries(document.getElementById("content"), toclevels);
for (var i = 0; i < entries.length; ++i) {
var entry = entries[i];
if (entry.element.id == "")
entry.element.id = "_toc_" + i;
var a = document.createElement("a");
a.href = "#" + entry.element.id;
a.appendChild(document.createTextNode(entry.text));
var div = document.createElement("div");
div.appendChild(a);
div.className = "toclevel" + entry.toclevel;
toc.appendChild(div);
}
if (entries.length == 0)
toc.parentNode.removeChild(toc);
},
/////////////////////////////////////////////////////////////////////
// Footnotes generator
/////////////////////////////////////////////////////////////////////
/* Based on footnote generation code from:
* http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
*/
footnotes: function () {
var cont = document.getElementById("content");
var noteholder = document.getElementById("footnotes");
var spans = cont.getElementsByTagName("span");
var refs = {};
var n = 0;
for (i=0; i<spans.length; i++) {
if (spans[i].className == "footnote") {
n++;
// Use [\s\S] in place of . so multi-line matches work.
// Because JavaScript has no s (dotall) regex flag.
note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
noteholder.innerHTML +=
"<div class='footnote' id='_footnote_" + n + "'>" +
"<a href='#_footnoteref_" + n + "' title='Return to text'>" +
n + "</a>. " + note + "</div>";
spans[i].innerHTML =
"[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
"' title='View footnote' class='footnote'>" + n + "</a>]";
var id =spans[i].getAttribute("id");
if (id != null) refs["#"+id] = n;
}
}
if (n == 0)
noteholder.parentNode.removeChild(noteholder);
else {
// Process footnoterefs.
for (i=0; i<spans.length; i++) {
if (spans[i].className == "footnoteref") {
var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
href = href.match(/#.*/)[0]; // Because IE return full URL.
n = refs[href];
spans[i].innerHTML =
"[<a href='#_footnote_" + n +
"' title='View footnote' class='footnote'>" + n + "</a>]";
}
}
}
}
}
/*]]>*/
</script>
</head>
<body style="max-width:750px">
<div id="header">
<h1>Futuregrid Quickstart Guide</h1>
<div id="toc">
<div id="toctitle">Table of Contents</div>
<noscript><p><b>JavaScript must be enabled in your browser to display the table of contents.</b></p></noscript>
</div>
</div>
<div id="content">
<h2 id="_downloading_and_building_swift">Downloading and Building Swift</h2>
<div class="sectionbody">
<div class="paragraph"><p>The most recent versions of Swift can be found at
<a href="http://www.ci.uchicago.edu/swift/downloads/index.php">http://www.ci.uchicago.edu/swift/downloads/index.php</a>. To run on futuregrid,
you will need to download the development/trunk version from SVN.</p></div>
</div>
<h2 id="_adding_swift_to_your_path">Adding Swift to your PATH</h2>
<div class="sectionbody">
<div class="paragraph"><p>Once you have installed Swift, add the Swift binary to your PATH so you can
easily run it from any directory.</p></div>
<div class="paragraph"><p>In your home directory, edit the file ".bashrc".</p></div>
<div class="paragraph"><p>If you have installed Swift via a source repository, add the following line
at the bottom of .bashrc.</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>export PATH=$PATH:$HOME/cog/modules/swift/dist/swift-svn/bin</tt></pre>
</div></div>
</div>
<h2 id="_requesting_futuregrid_access">Requesting Futuregrid Access</h2>
<div class="sectionbody">
<div class="paragraph"><p>If you do not already have a futuregrid account, you can follow the
instructions at <a href="https://portal.futuregrid.org/gettingstarted">https://portal.futuregrid.org/gettingstarted</a> to get started.
This page provides information on how to create an account, how to join
a project, how to set up your SSH keys, and how to create a new project.</p></div>
</div>
<h2 id="_downloading_swift_vm_tools">Downloading Swift VM Tools</h2>
<div class="sectionbody">
<div class="paragraph"><p>A set of scripts based around cloudinitd are used to easily start virtual
machines. To download, change to your home directory and run the
following command:</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>$ svn co https://svn.ci.uchicago.edu/svn/vdl2/usertools/swift-vm-boot</tt></pre>
</div></div>
</div>
<h2 id="_download_your_credentials">Download your Credentials</h2>
<div class="sectionbody">
<div class="paragraph"><p>Run the following commands to retrieve your credentials:</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>$ scp yourusername@hotel.futuregrid.org:nimbus_creds.tar.gz .
$ tar xvfz nimbus_creds.tar.gz</tt></pre>
</div></div>
<div class="paragraph"><p>When you extract your credential file, look at the file called
hotel.conf. Near the bottom of this file will be two settings
called vws.repository.s3id and vws.repository.s3key. Copy these
values for the next step.</p></div>
</div>
<h2 id="_configuring_swift_vm_tools">Configuring Swift VM Tools</h2>
<div class="sectionbody">
<div class="paragraph"><p>Change directories into the swift-vm-boot directory. Edit the file
called env.sh. There are two settings here that you will need to
modify: FUTUREGRID_IAAS_ACCESS_KEY and FUTUREGRID_IAAS_SECRET_KEY.
Paste your access key (s3id) and secret key (s3key) here and save
the file. By default, env.sh requests 2 nodes on hotel and 2 nodes
on sierra. Change these values as needed. Run install.sh from the
swift-vm-boot directory to complete the installation of Swift VM
Tools. The Swift VM Tools may require a fairly recent version of
Python. If you run into problems while running the install.sh
script, please try a more recent version of Python and associated
libraries.</p></div>
</div>
<h2 id="_configuring_coaster_service_conf">Configuring coaster-service.conf</h2>
<div class="sectionbody">
<div class="paragraph"><p>Within your swift distribution’s etc directory is a file called
coaster-service.conf. There are many options here you can modify
as needed, but these are the settings that will be required
to run on futuregrid:</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>export WORKER_WORK=/tmp
export WORKER_MODE=futuregrid
export SHARED_FILESYSTEM=no
export WORKER_USERNAME=root
export SSH_TUNNELING=yes
export SWIFTVMBOOT_DIR=$HOME/swift-vm-boot
export WORK=/tmp</tt></pre>
</div></div>
<div class="paragraph"><p>These is a brief description of these settings in the coaster-service.conf file.
Tunneling should be used when you are restricted by a firewall. If your local
machine has multiple network interfaces, you should also manually set the IP
address of your machine with export IPADDR=your.ip.address.here.</p></div>
</div>
<h2 id="_starting_the_coaster_service_script">Starting the Coaster Service Script</h2>
<div class="sectionbody">
<div class="paragraph"><p>Now that everything is configured, change to a temporary directory
and run this command to start the coaster service:</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>$ start-coaster-service</tt></pre>
</div></div>
<div class="paragraph"><p>This command will start the VMs, start the required processes on the worker nodes,
and generate Swift configuration files for you to use. The configuration files
will be generated in your current directory. These files are sites.xml, tc.data,
and cf.</p></div>
</div>
<h2 id="_running_swift">Running Swift</h2>
<div class="sectionbody">
<div class="paragraph"><p>Now that you have all of your configuration files generated, run the following command:</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>$ swift -sites.file sites.xml -tc.file tc.data -config cf <yourscript.swift></tt></pre>
</div></div>
<div class="paragraph"><p>If you would like to create a custom tc file for repeated use, rename it to something other
than tc.data to prevent it from being overwritten. The sites.xml however will need to be
regenerated every time you start the coaster service. If you need to repeatedly modify some
sites.xml options, you may edit the template in Swift’s etc/sites/persistent-coasters. You
may also create your own custom tc files with the hostname of persistent-coasters. More
information about this can be found in the Swift userguide at
<a href="http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html">http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html</a>.</p></div>
</div>
<h2 id="_stopping_the_coaster_service_script">Stopping the Coaster Service Script</h2>
<div class="sectionbody">
<div class="paragraph"><p>To stop the coaster service, run the following command:</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>$ stop-coaster-service</tt></pre>
</div></div>
<div class="paragraph"><p>This will kill the coaster service, kill the worker scripts on remote systems and terminate
the virtual machines that were created during start-coaster-service.</p></div>
</div>
<h2 id="_more_help">More Help</h2>
<div class="sectionbody">
<div class="paragraph"><p>The best place for additional help is the Swift user mailing list. You can subscribe to this list at
<a href="http://mail.ci.uchicago.edu/mailman/listinfo/swift-user">http://mail.ci.uchicago.edu/mailman/listinfo/swift-user</a>. When submitting information, please send
your sites.xml file, your tc.data, and any error messages you run into.</p></div>
</div>
</div>
<div id="footnotes"><hr /></div>
<div id="footer">
<div id="footer-text">
Last updated 2011-06-07 10:13:12 CDT
</div>
</div>
</body>
</html>