[Swift-commit] r4427 - www/cookbook
ketan at ci.uchicago.edu
ketan at ci.uchicago.edu
Wed Apr 27 11:10:09 CDT 2011
Author: ketan
Date: 2011-04-27 11:10:09 -0500 (Wed, 27 Apr 2011)
New Revision: 4427
Modified:
www/cookbook/cookbook-asciidoc.html
www/cookbook/cookbook-asciidoc.txt
Log:
Modified: www/cookbook/cookbook-asciidoc.html
===================================================================
--- www/cookbook/cookbook-asciidoc.html 2011-04-27 02:35:08 UTC (rev 4426)
+++ www/cookbook/cookbook-asciidoc.html 2011-04-27 16:10:09 UTC (rev 4427)
@@ -581,11 +581,12 @@
<div class="sect1">
<h2 id="_overview">1. Overview</h2>
<div class="sectionbody">
-<div class="paragraph"><p>This cookbook covers various recipes involving running Swift under diverse
-configurations based on the application requirements and underlying
-infrastructures. The SwiftScript language and the Swift runtim system. For
-introductory material, consult the Swift tutorial.</p></div>
-<div class="paragraph"><p>Swift is a data-oriented coarse grained scripting language that supports
+<div class="paragraph"><p>This cookbook covers various recipes involving setting up and running Swift under diverse
+configurations based on the application requirements and the underlying
+infrastructures. The Swift system comprises of SwiftScript language and the Swift runtime system. For
+introductory material, consult the Swift tutorial found
+<a href="http://www.ci.uchicago.edu/swift/guides/tutorial.php">here</a>.</p></div>
+<div class="paragraph"><p>Swift is a data-oriented coarse grained parallel scripting language that supports
dataset typing and mapping, dataset iteration, conditional branching, and
procedural composition.</p></div>
<div class="paragraph"><p>Swift programs (or workflows) are written in a language called SwiftScript.</p></div>
@@ -632,10 +633,17 @@
<div class="content">
<pre><tt>PATH=$PATH:/home/wilde/bigdata/swift/bin</tt></pre>
</div></div>
-<div class="paragraph"><p>Do NOT set SWIFT_HOME or CLASSPATH in your environment unless you fully
-understand how these will affect Swift’s execution.</p></div>
+<div class="admonitionblock">
+<table><tr>
+<td class="icon">
+<div class="title">Warning</div>
+</td>
+<td class="content">Do NOT set SWIFT_HOME or CLASSPATH in your environment unless you fully
+understand how these will affect Swift’s execution.</td>
+</tr></table>
</div>
</div>
+</div>
<div class="sect2">
<h3 id="_environment_setup">2.2. Environment Setup</h3>
<div class="paragraph"><p>Setting up the environment</p></div>
@@ -645,19 +653,14 @@
<div class="title">Note</div>
</td>
<td class="content">The environment will be different when using Swift from prebuilt
-distribution and trunk. To execute your Swift script on the login host ("localhost") use this
-command:</td>
+distribution and trunk. To execute your Swift script on a login host (or "localhost") use the
+following command:</td>
</tr></table>
</div>
<div class="listingblock">
<div class="content">
-<pre><tt>swift -tc.file tc modis.swift</tt></pre>
+<pre><tt>swift -tc.file tc somescript.swift</tt></pre>
</div></div>
-<div class="paragraph"><p>To execute your Swift script on the PADS cluster use this command:</p></div>
-<div class="listingblock">
-<div class="content">
-<pre><tt>swift -tc.file tc -sites.file pbs.xml modis.swift</tt></pre>
-</div></div>
<div class="sect3">
<h4 id="_setting_transformation_catalog">2.2.1. Setting transformation catalog</h4>
<div class="paragraph"><p>tc</p></div>
@@ -1010,7 +1013,33 @@
will get fast prioritized execution. Good for small tests</p></div>
</div>
<div class="sect2">
-<h3 id="_osg">3.2. OSG</h3>
+<h3 id="_pads">3.2. PADS</h3>
+<div class="paragraph"><p>Swift on PADS
+To execute your Swift script on the PADS cluster use this command:</p></div>
+<div class="listingblock">
+<div class="content">
+<pre><tt>swift -tc.file tc -sites.file pbs.xml modis.swift</tt></pre>
+</div></div>
+<div class="paragraph"><p>where the contents of a simple pbs.xml sites file could be:</p></div>
+<div class="listingblock">
+<div class="content"><!-- Generator: GNU source-highlight 3.1.4
+by Lorenzo Bettini
+http://www.lorenzobettini.it
+http://www.gnu.org/software/src-highlite -->
+<pre><tt><span style="font-weight: bold"><span style="color: #0000FF"><config></span></span>
+ <span style="font-weight: bold"><span style="color: #0000FF"><pool</span></span> <span style="color: #009900">handle</span><span style="color: #990000">=</span><span style="color: #FF0000">"pbs"</span><span style="font-weight: bold"><span style="color: #0000FF">></span></span>
+ <span style="font-weight: bold"><span style="color: #0000FF"><execution</span></span> <span style="color: #009900">provider</span><span style="color: #990000">=</span><span style="color: #FF0000">"pbs"</span> <span style="color: #009900">url</span><span style="color: #990000">=</span><span style="color: #FF0000">"none"</span><span style="font-weight: bold"><span style="color: #0000FF">/></span></span>
+ <span style="font-weight: bold"><span style="color: #0000FF"><profile</span></span> <span style="color: #009900">namespace</span><span style="color: #990000">=</span><span style="color: #FF0000">"globus"</span> <span style="color: #009900">key</span><span style="color: #990000">=</span><span style="color: #FF0000">"queue"</span><span style="font-weight: bold"><span style="color: #0000FF">></span></span>fast<span style="font-weight: bold"><span style="color: #0000FF"></profile></span></span>
+ <span style="font-weight: bold"><span style="color: #0000FF"><profile</span></span> <span style="color: #009900">namespace</span><span style="color: #990000">=</span><span style="color: #FF0000">"globus"</span> <span style="color: #009900">key</span><span style="color: #990000">=</span><span style="color: #FF0000">"maxwalltime"</span><span style="font-weight: bold"><span style="color: #0000FF">></span></span>00:05:00<span style="font-weight: bold"><span style="color: #0000FF"></profile></span></span>
+ <span style="font-weight: bold"><span style="color: #0000FF"><profile</span></span> <span style="color: #009900">namespace</span><span style="color: #990000">=</span><span style="color: #FF0000">"karajan"</span> <span style="color: #009900">key</span><span style="color: #990000">=</span><span style="color: #FF0000">"initialScore"</span><span style="font-weight: bold"><span style="color: #0000FF">></span></span>10000<span style="font-weight: bold"><span style="color: #0000FF"></profile></span></span>
+ <span style="font-weight: bold"><span style="color: #0000FF"><profile</span></span> <span style="color: #009900">namespace</span><span style="color: #990000">=</span><span style="color: #FF0000">"karajan"</span> <span style="color: #009900">key</span><span style="color: #990000">=</span><span style="color: #FF0000">"jobThrottle"</span><span style="font-weight: bold"><span style="color: #0000FF">></span></span>.10<span style="font-weight: bold"><span style="color: #0000FF"></profile></span></span>
+ <span style="font-weight: bold"><span style="color: #0000FF"><filesystem</span></span> <span style="color: #009900">provider</span><span style="color: #990000">=</span><span style="color: #FF0000">"local"</span><span style="font-weight: bold"><span style="color: #0000FF">/></span></span>
+ <span style="font-weight: bold"><span style="color: #0000FF"><workdirectory</span></span> <span style="font-weight: bold"><span style="color: #0000FF">></span></span>/home/you/swiftwork<span style="font-weight: bold"><span style="color: #0000FF"></workdirectory></span></span>
+ <span style="font-weight: bold"><span style="color: #0000FF"></pool></span></span>
+<span style="font-weight: bold"><span style="color: #0000FF"></config></span></span></tt></pre></div></div>
+</div>
+<div class="sect2">
+<h3 id="_osg">3.3. OSG</h3>
<div class="imageblock">
<div class="content">
<img src="figures/coaster_setup.png" alt="Coaster setup" />
@@ -1019,7 +1048,7 @@
</div>
</div>
<div class="sect2">
-<h3 id="_bionimbus">3.3. Bionimbus</h3>
+<h3 id="_bionimbus">3.4. Bionimbus</h3>
<div class="paragraph"><p>This section explains a step by step procedure on getting Swift running on the Bionimbus cloud. We will use the <em>manual</em> <em>coasters</em> configuration on the Bionimbus cloud.</p></div>
<div class="paragraph"><p><strong>step1.</strong> Connect to the gateway (ssh gatewayx.lac.uic.edu)</p></div>
<div class="paragraph"><p><strong>step2.</strong> Start a virtual machine (euca-run-instances -n 1 -t m1.small
@@ -1339,7 +1368,7 @@
<div id="footer">
<div id="footer-text">
Version 0.92<br />
-Last updated 2011-04-26 11:50:24 CDT
+Last updated 2011-04-27 11:09:55 CDT
</div>
</div>
</body>
Modified: www/cookbook/cookbook-asciidoc.txt
===================================================================
--- www/cookbook/cookbook-asciidoc.txt 2011-04-27 02:35:08 UTC (rev 4426)
+++ www/cookbook/cookbook-asciidoc.txt 2011-04-27 16:10:09 UTC (rev 4427)
@@ -8,12 +8,13 @@
Overview
--------
-This cookbook covers various recipes involving running Swift under diverse
-configurations based on the application requirements and underlying
-infrastructures. The SwiftScript language and the Swift runtim system. For
-introductory material, consult the Swift tutorial.
+This cookbook covers various recipes involving setting up and running Swift under diverse
+configurations based on the application requirements and the underlying
+infrastructures. The Swift system comprises of SwiftScript language and the Swift runtime system. For
+introductory material, consult the Swift tutorial found
+link:http://www.ci.uchicago.edu/swift/guides/tutorial.php[here].
-Swift is a data-oriented coarse grained scripting language that supports
+Swift is a data-oriented coarse grained parallel scripting language that supports
dataset typing and mapping, dataset iteration, conditional branching, and
procedural composition.
@@ -68,29 +69,22 @@
PATH=$PATH:/home/wilde/bigdata/swift/bin
----
-Do NOT set SWIFT_HOME or CLASSPATH in your environment unless you fully
+WARNING: Do NOT set SWIFT_HOME or CLASSPATH in your environment unless you fully
understand how these will affect Swift's execution.
-
Environment Setup
~~~~~~~~~~~~~~~~~
Setting up the environment
[NOTE]
The environment will be different when using Swift from prebuilt
-distribution and trunk. To execute your Swift script on the login host ("localhost") use this
-command:
+distribution and trunk. To execute your Swift script on a login host (or "localhost") use the
+following command:
----
-swift -tc.file tc modis.swift
+swift -tc.file tc somescript.swift
----
-To execute your Swift script on the PADS cluster use this command:
-
-----
-swift -tc.file tc -sites.file pbs.xml modis.swift
-----
-
Setting transformation catalog
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -427,10 +421,31 @@
//~~~~~~~~~~~~~
//Swift on Intrepid-BG/P
//
-//PADS
-//~~~~
-//Swift on PADS
-//
+
+PADS
+~~~~
+Swift on PADS
+To execute your Swift script on the PADS cluster use this command:
+
+----
+swift -tc.file tc -sites.file pbs.xml modis.swift
+----
+where the contents of a simple pbs.xml sites file could be:
+[xml]
+source~~~~
+<config>
+ <pool handle="pbs">
+ <execution provider="pbs" url="none"/>
+ <profile namespace="globus" key="queue">fast</profile>
+ <profile namespace="globus" key="maxwalltime">00:05:00</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <profile namespace="karajan" key="jobThrottle">.10</profile>
+ <filesystem provider="local"/>
+ <workdirectory >/home/you/swiftwork</workdirectory>
+ </pool>
+</config>
+source~~~~
+
OSG
~~~
.Coaster setup on OSG
More information about the Swift-commit
mailing list