<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="generator" content="AsciiDoc 8.5.2" />
<title>Coaster Service Quickstart</title>
<style type="text/css">
/* Debug borders */
p, li, dt, dd, div, pre, h1, h2, h3, h4, h5, h6 {
/*
border: 1px solid red;
*/
}
body {
margin: 1em 5% 1em 5%;
}
a {
color: blue;
text-decoration: underline;
}
a:visited {
color: fuchsia;
}
em {
font-style: italic;
color: navy;
}
strong {
font-weight: bold;
color: #083194;
}
tt {
color: navy;
}
h1, h2, h3, h4, h5, h6 {
color: #527bbd;
font-family: sans-serif;
margin-top: 1.2em;
margin-bottom: 0.5em;
line-height: 1.3;
}
h1, h2, h3 {
border-bottom: 2px solid silver;
}
h2 {
padding-top: 0.5em;
}
h3 {
float: left;
}
h3 + * {
clear: left;
}
div.sectionbody {
font-family: serif;
margin-left: 0;
}
hr {
border: 1px solid silver;
}
p {
margin-top: 0.5em;
margin-bottom: 0.5em;
}
ul, ol, li > p {
margin-top: 0;
}
pre {
padding: 0;
margin: 0;
}
span#author {
color: #527bbd;
font-family: sans-serif;
font-weight: bold;
font-size: 1.1em;
}
span#email {
}
span#revnumber, span#revdate, span#revremark {
font-family: sans-serif;
}
div#footer {
font-family: sans-serif;
font-size: small;
border-top: 2px solid silver;
padding-top: 0.5em;
margin-top: 4.0em;
}
div#footer-text {
float: left;
padding-bottom: 0.5em;
}
div#footer-badges {
float: right;
padding-bottom: 0.5em;
}
div#preamble {
margin-top: 1.5em;
margin-bottom: 1.5em;
}
div.tableblock, div.imageblock, div.exampleblock, div.verseblock,
div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
div.admonitionblock {
margin-top: 1.0em;
margin-bottom: 1.5em;
}
div.admonitionblock {
margin-top: 2.0em;
margin-bottom: 2.0em;
margin-right: 10%;
color: #606060;
}
div.content { /* Block element content. */
padding: 0;
}
/* Block element titles. */
div.title, caption.title {
color: #527bbd;
font-family: sans-serif;
font-weight: bold;
text-align: left;
margin-top: 1.0em;
margin-bottom: 0.5em;
}
div.title + * {
margin-top: 0;
}
td div.title:first-child {
margin-top: 0.0em;
}
div.content div.title:first-child {
margin-top: 0.0em;
}
div.content + div.title {
margin-top: 0.0em;
}
div.sidebarblock > div.content {
background: #ffffee;
border: 1px solid silver;
padding: 0.5em;
}
div.listingblock > div.content {
border: 1px solid silver;
background: #f4f4f4;
padding: 0.5em;
}
div.quoteblock, div.verseblock {
padding-left: 1.0em;
margin-left: 1.0em;
margin-right: 10%;
border-left: 5px solid #dddddd;
color: #777777;
}
div.quoteblock > div.attribution {
padding-top: 0.5em;
text-align: right;
}
div.verseblock > div.content {
white-space: pre;
}
div.verseblock > div.attribution {
padding-top: 0.75em;
text-align: left;
}
/* DEPRECATED: Pre version 8.2.7 verse style literal block. */
div.verseblock + div.attribution {
text-align: left;
}
div.admonitionblock .icon {
vertical-align: top;
font-size: 1.1em;
font-weight: bold;
text-decoration: underline;
color: #527bbd;
padding-right: 0.5em;
}
div.admonitionblock td.content {
padding-left: 0.5em;
border-left: 3px solid #dddddd;
}
div.exampleblock > div.content {
border-left: 3px solid #dddddd;
padding-left: 0.5em;
}
div.imageblock div.content { padding-left: 0; }
span.image img { border-style: none; }
a.image:visited { color: white; }
dl {
margin-top: 0.8em;
margin-bottom: 0.8em;
}
dt {
margin-top: 0.5em;
margin-bottom: 0;
font-style: normal;
color: navy;
}
dd > *:first-child {
margin-top: 0.1em;
}
ul, ol {
list-style-position: outside;
}
ol.arabic {
list-style-type: decimal;
}
ol.loweralpha {
list-style-type: lower-alpha;
}
ol.upperalpha {
list-style-type: upper-alpha;
}
ol.lowerroman {
list-style-type: lower-roman;
}
ol.upperroman {
list-style-type: upper-roman;
}
div.compact ul, div.compact ol,
div.compact p, div.compact p,
div.compact div, div.compact div {
margin-top: 0.1em;
margin-bottom: 0.1em;
}
div.tableblock > table {
border: 3px solid #527bbd;
}
thead, p.table.header {
font-family: sans-serif;
font-weight: bold;
}
tfoot {
font-weight: bold;
}
td > div.verse {
white-space: pre;
}
p.table {
margin-top: 0;
}
/* Because the table frame attribute is overriden by CSS in most browsers. */
div.tableblock > table[frame="void"] {
border-style: none;
}
div.tableblock > table[frame="hsides"] {
border-left-style: none;
border-right-style: none;
}
div.tableblock > table[frame="vsides"] {
border-top-style: none;
border-bottom-style: none;
}
div.hdlist {
margin-top: 0.8em;
margin-bottom: 0.8em;
}
div.hdlist tr {
padding-bottom: 15px;
}
dt.hdlist1.strong, td.hdlist1.strong {
font-weight: bold;
}
td.hdlist1 {
vertical-align: top;
font-style: normal;
padding-right: 0.8em;
color: navy;
}
td.hdlist2 {
vertical-align: top;
}
div.hdlist.compact tr {
margin: 0;
padding-bottom: 0;
}
.comment {
background: yellow;
}
.footnote, .footnoteref {
font-size: 0.8em;
}
span.footnote, span.footnoteref {
vertical-align: super;
}
#footnotes {
margin: 20px 0 20px 0;
padding: 7px 0 0 0;
}
#footnotes div.footnote {
margin: 0 0 5px 0;
}
#footnotes hr {
border: none;
border-top: 1px solid silver;
height: 1px;
text-align: left;
margin-left: 0;
width: 20%;
min-width: 100px;
}
@media print {
div#footer-badges { display: none; }
}
div#toc {
margin-bottom: 2.5em;
}
div#toctitle {
color: #527bbd;
font-family: sans-serif;
font-size: 1.1em;
font-weight: bold;
margin-top: 1.0em;
margin-bottom: 0.1em;
}
div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
margin-top: 0;
margin-bottom: 0;
}
div.toclevel2 {
margin-left: 2em;
font-size: 0.9em;
}
div.toclevel3 {
margin-left: 4em;
font-size: 0.9em;
}
div.toclevel4 {
margin-left: 6em;
font-size: 0.9em;
}
/* Workarounds for IE6's broken and incomplete CSS2. */
div.sidebar-content {
background: #ffffee;
border: 1px solid silver;
padding: 0.5em;
}
div.sidebar-title, div.image-title {
color: #527bbd;
font-family: sans-serif;
font-weight: bold;
margin-top: 0.0em;
margin-bottom: 0.5em;
}
div.listingblock div.content {
border: 1px solid silver;
background: #f4f4f4;
padding: 0.5em;
}
div.quoteblock-attribution {
padding-top: 0.5em;
text-align: right;
}
div.verseblock-content {
white-space: pre;
}
div.verseblock-attribution {
padding-top: 0.75em;
text-align: left;
}
div.exampleblock-content {
border-left: 3px solid #dddddd;
padding-left: 0.5em;
}
/* IE6 sets dynamically generated links as visited. */
div#toc a:visited { color: blue; }
</style>
<script type="text/javascript">
/*<+'])');
// Function that scans the DOM tree for header elements (the DOM2
// nodeIterator API would be a better technique but not supported by all
// browsers).
var iterate = function (el) {
for (var i = el.firstChild; i != null; i = i.nextSibling) {
if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
var mo = re.exec(i.tagName);
if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
}
iterate(i);
}
}
}
iterate(el);
return result;
}
var toc = document.getElementById("toc");
var entries = tocEntries(document.getElementById("content"), toclevels);
for (var i = 0; i < entries.length; ++i) {
var entry = entries[i];
if (entry.element.id == "")
entry.element.id = "_toc_" + i;
var a = document.createElement("a");
a.href = "#" + entry.element.id;
a.appendChild(document.createTextNode(entry.text));
var div = document.createElement("div");
div.appendChild(a);
div.className = "toclevel" + entry.toclevel;
toc.appendChild(div);
}
if (entries.length == 0)
toc.parentNode.removeChild(toc);
},
/////////////////////////////////////////////////////////////////////
// Footnotes generator
/////////////////////////////////////////////////////////////////////
/* Based on footnote generation code from:
* http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
*/
footnotes: function () {
var cont = document.getElementById("content");
var noteholder = document.getElementById("footnotes");
var spans = cont.getElementsByTagName("span");
var refs = {};
var n = 0;
for (i=0; i<spans.length; i++) {
if (spans[i].className == "footnote") {
n++;
// Use [\s\S] in place of . so multi-line matches work.
// Because JavaScript has no s (dotall) regex flag.
note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
noteholder.innerHTML +=
"<div class='footnote' id='_footnote_" + n + "'>" +
"<a href='#_footnoteref_" + n + "' title='Return to text'>" +
n + "</a>. " + note + "</div>";
spans[i].innerHTML =
"[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
"' title='View footnote' class='footnote'>" + n + "</a>]";
var id =spans[i].getAttribute("id");
if (id != null) refs["#"+id] = n;
}
}
if (n == 0)
noteholder.parentNode.removeChild(noteholder);
else {
// Process footnoterefs.
for (i=0; i<spans.length; i++) {
if (spans[i].className == "footnoteref") {
var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
href = href.match(/#.*/)[0]; // Because IE return full URL.
n = refs[href];
spans[i].innerHTML =
"[<a href='#_footnote_" + n +
"' title='View footnote' class='footnote'>" + n + "</a>]";
}
}
}
}
}
/*]]>*/
</script>
</head>
<body style="max-width:750px">
<div id="header">
<h1>Coaster Service Quickstart</h1>
<div id="toc">
<div id="toctitle">Table of Contents</div>
<noscript><p><b>JavaScript must be enabled in your browser to display the table of contents.</b></p></noscript>
</div>
</div>
<div id="content">
<h2 id="_persistent_coasters">Persistent Coasters</h2>
<div class="sectionbody">
<div class="paragraph"><p>There are three main parts to the persistent coasters scripts:
coaster-service.conf, start-coaster-service, and stop-coaster-service.
The coaster-service.conf file is a configuration file that determines how
coasters should run. Start-coaster-service and stop-coaster-service start
and stop the processes that are needed to run Swift in this setup.</p></div>
<h3 id="_coaster_service_conf">coaster-service.conf</h3><div style="clear:left"></div>
<div class="paragraph"><p>The coaster-service.conf file is used to control the way
the coaster service runs. It is located in the etc/ directory
of your Swift distribution. Below is an explanation of what
the settings in coaster-service.conf do.</p></div>
<h4 id="_ip_address">IP Address</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export IPADDR=123.456.7.8</tt></pre>
</div></div>
<div class="paragraph"><p>This setting is the IP address of the machine on which you are running start-coaster-service.
When empty, start-coaster-service will attempt to automatically determine the IP address.
In the cases of machines with multiple network devices, it may be better to specify the address
here.</p></div>
<h4 id="_local_port_and_service_port">Local Port and Service Port</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export LOCAL_PORT=1984
export SERVICE_PORT=2001</tt></pre>
</div></div>
<div class="paragraph"><p>The local and service ports are the port numbers that are used by workers and by swift
to communicate with each other. When these ports are undefined, coaster-service will
automatically find and use an available port.</p></div>
<h4 id="_log_directory">Log Directory</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export LOG_DIR=logs</tt></pre>
</div></div>
<div class="paragraph"><p>This is the directory to store log files, relative to your current working directory.
Currently there are two log files that are stored here: swift.log - the output of the swift
process, and coasters.log, the output of coaster-service.</p></div>
<h4 id="_project">Project</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export PROJECT=CI-CCR000013</tt></pre>
</div></div>
<div class="paragraph"><p>Name of project when submitting to scheduler.</p></div>
<h4 id="_queue">Queue</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export QUEUE=fast</tt></pre>
</div></div>
<div class="paragraph"><p>When working directly with a scheduler, specify a queue to where work will be submitted.
used to specify an internal hostname in sites.xml for systems with multiple
interfaces.</p></div>
<h4 id="_shared_filesystem">Shared Filesystem</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export SHARED_FILESYSTEM=yes</tt></pre>
</div></div>
<div class="paragraph"><p>Valid values for SHARED_FILESYSTEM are either yes or no. This defines
if all the nodes involved share a common filesystem. By default
Swift assumes this is true.</p></div>
<div class="paragraph"><p>When SHARED_FILESYSTEM is set to no, a Swift configuration file called
cf is generated. It contains information that turns on provider staging,
a mechanism for running on machines without a shared filesystems.</p></div>
<h4 id="_ssh_tunneling">SSH Tunneling</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export SSH_TUNNNELING=yes</tt></pre>
</div></div>
<div class="paragraph"><p>SSH tunneling provides a way to bypass restrictive firewalls. Use this is there is
a firewall blocking ports between the head node and worker nodes.</p></div>
<h4 id="_swift_path">Swift path</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export SWIFT=/home/davidk/swift</tt></pre>
</div></div>
<div class="paragraph"><p>The start-coaster-service script will automatically try to
determine the path to Swift. If you see any error messages
about not being able to find swift, you can manually specify
the full path to swift here.</p></div>
<h4 id="_swift_vm_boot_directory">Swift VM Boot Directory</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export SWIFTVMBOOT_DIR=$HOME/swift-vm-boot</tt></pre>
</div></div>
<div class="paragraph"><p>When configured to work with Futuregrid, this setting will determine the location of the swift-vm-boot
utility which is responsible for requesting virtual machines.</p></div>
<h4 id="_work_directory">Work Directory</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export WORK=$HOME/work</tt></pre>
</div></div>
<div class="paragraph"><p>This setting defines a directory in which Swift can write to while it executes work.</p></div>
<h4 id="_worker_hosts">Worker Hosts</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export WORKER_HOSTS="host1 host2 host3"</tt></pre>
</div></div>
<div class="paragraph"><p>Worker hosts is a list of hosts to start workers on when in SSH mode.</p></div>
<h4 id="_worker_mode">Worker Mode</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export WORKER_MODE=ssh</tt></pre>
</div></div>
<div class="paragraph"><p>Worker mode describes how to start the needed workers.
Valid options include ssh, local, cobalt, and futuregrid.</p></div>
<div class="paragraph"><p>SSH mode requires having keys on all the worker nodes
that will allow you to log in without prompting. SSH
is used to handle starting and stopping workers.</p></div>
<div class="paragraph"><p>Local mode starts a single worker on the local machine.</p></div>
<div class="paragraph"><p>Cobalt mode starts a worker process via the cobalt job
management suite</p></div>
<div class="paragraph"><p>Futuregrid works similarly to SSH mode, but will also start
and stop instances of virtual machines.</p></div>
<h4 id="_worker_username">Worker Username</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export WORKER_USERNAME=david</tt></pre>
</div></div>
<div class="paragraph"><p>In modes that use SSH, modify this value to log in as a different user on worker nodes.
By default this value is set to $USER and assumes login names are the same on each
system.</p></div>
<h4 id="_worker_work_directory">Worker Work Directory</h4>
<div class="listingblock">
<div class="content">
<pre><tt>export WORKER_WORK=/home/${USER}/work</tt></pre>
</div></div>
<div class="paragraph"><p>This setting should point to a directory on the worker nodes that
is writable to you. This is a reasonable default, but you may
need to change it in cases where your worker node has a different
directory structure.</p></div>
<h3 id="_start_coaster_service">start-coaster-service</h3><div style="clear:left"></div>
<div class="paragraph"><p>Once coaster-service.conf is configured, the start-coaster-service script
will take care of everything else. It will start the coaster service with
the correct arguments, start SSH tunneling, and start the worker.pl script
on all worker nodes. It will create a sites.xml and tc.data in your
current working directory. If shared filesystem is set to no, it will also
generate a Swift configuration file called cf. This contains the settings
needed for provider staging. The start-coaster-service script does not take
any arguments. It is located in Swift’s bin directory.</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>$ start-coaster-service</tt></pre>
</div></div>
<h3 id="_running_swift">Running Swift</h3><div style="clear:left"></div>
<div class="paragraph"><p>Now that start-coaster-service has started all the required programs and
generated Swift configuration files, you must tell Swift to use them via
command line arguments.</p></div>
<div class="listingblock">
<div class="title">Running Swift (with SHARED_FILESYSTEM=yes)</div>
<div class="content">
<pre><tt>$ swift -sites.file sites.xml -tc.file tc.data <myscript.swift></tt></pre>
</div></div>
<div class="listingblock">
<div class="title">Running Swift (with SHARED_FILESYSTEM=no)</div>
<div class="content">
<pre><tt>$ swift -sites.file sites.xml -tc.file tc.data -config cf <myscript.swift></tt></pre>
</div></div>
<h3 id="_stop_coaster_service">stop-coaster-service</h3><div style="clear:left"></div>
<div class="paragraph"><p>The stop-coaster-service script will stop all of the services started by
start-coaster-service. It ends the coaster service, worker scripts and
SSH tunneling. The stop-coaster-service takes no command line arguments.</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>$ stop-coaster-service</tt></pre>
</div></div>
</div>
</div>
<div id="footnotes"><hr /></div>
<div id="footer">
<div id="footer-text">
Last updated 2011-06-08 09:10:32 CDT
</div>
</div>
</body>
</html>