[Swift-commit] r4373 - www/cookbook
ketan at ci.uchicago.edu
ketan at ci.uchicago.edu
Wed Apr 13 17:55:14 CDT 2011
Author: ketan
Date: 2011-04-13 17:55:14 -0500 (Wed, 13 Apr 2011)
New Revision: 4373
Modified:
www/cookbook/cookbook-asciidoc.html
www/cookbook/cookbook-asciidoc.txt
Log:
Added bionimbus-swift related stuff, to be refined and organized
Modified: www/cookbook/cookbook-asciidoc.html
===================================================================
--- www/cookbook/cookbook-asciidoc.html 2011-04-13 20:41:48 UTC (rev 4372)
+++ www/cookbook/cookbook-asciidoc.html 2011-04-13 22:55:14 UTC (rev 4373)
@@ -585,8 +585,7 @@
<div class="paragraph"><p>This cookbook covers various recipes involving running Swift under diverse configurations based on the application requirements and underlying infrastructures. The SwiftScript language and the Swift runtim system. For introductory material, consult the Swift tutorial.</p></div>
<div class="paragraph"><p>Swift is a data-oriented coarse grained scripting language that supports dataset typing and mapping, dataset iteration, conditional branching, and procedural composition.</p></div>
<div class="paragraph"><p>Swift programs (or workflows) are written in a language called SwiftScript.</p></div>
-<div class="paragraph"><p>SwiftScript programs are dataflow oriented - they are primarily concerned with processing (possibly large) collections of data files, by invoking programs to do that processing. Swift handles execution of
-such programs on remote sites by choosing sites, handling the staging of input and output files to and from the chosen sites and remote execution of program code.</p></div>
+<div class="paragraph"><p>SwiftScript programs are dataflow oriented - they are primarily concerned with processing (possibly large) collections of data files, by invoking programs to do that processing. Swift handles execution of such programs on remote sites by choosing sites, handling the staging of input and output files to and from the chosen sites and remote execution of program code.</p></div>
</div>
</div>
<div class="sect1">
@@ -608,14 +607,22 @@
distribution and trunk.</td>
</tr></table>
</div>
+<div class="sect3">
+<h4 id="_setting_transformation_catalog">2.2.1. Setting transformation catalog</h4>
+<div class="paragraph"><p>tc</p></div>
</div>
+<div class="sect3">
+<h4 id="_setting_swift_configuration">2.2.2. Setting swift configuration</h4>
+<div class="paragraph"><p>cf</p></div>
+</div>
+</div>
<div class="sect2">
<h3 id="_first_swiftscript">2.3. First SwiftScript</h3>
<div class="paragraph"><p>Your first SwiftScript
Hello Swift-World!</p></div>
</div>
<div class="sect2">
-<h3 id="_second_swiftscript">2.4. Second SwiftScript</h3>
+<h3 id="_second_swiftscript">2.4. second SwiftScript</h3>
<div class="paragraph"><p>Your second SwiftScript
catsn</p></div>
</div>
@@ -658,7 +665,26 @@
</div>
<div class="sect2">
<h3 id="_bionimbus">3.5. Bionimbus</h3>
-<div class="paragraph"><p>Swift on Bionimbus</p></div>
+<div class="paragraph"><p>Swift on Bionimbus
+.VERY IMPORTANT TO READ THIS</p></div>
+<div class="paragraph"><p>CONTAINS INFORMATION ON HOW TO CONNECT BACK TO GATEWAY FROM VIRTUAL MACHINES
+USING REVERSE SSH TUNNELING</p></div>
+<div class="paragraph"><p>TO OPEN A REVERSE SSH TUNNEL DO THE FOLLOWING:
+.FROM THE GATEWAY PROMPT</p></div>
+<div class="paragraph"><p><tt>ssh -R *:5000:localhost:5000 <a href="mailto:root at 10.101.8.50">root at 10.101.8.50</a> sleep 999</tt></p></div>
+<div class="paragraph"><p>WHERE:
+*=network interface, should remain the same on all cases
+localhost=the gateway host, should remain the same</p></div>
+<div class="paragraph"><p>5000(LEFT OF localhost)=the port number on localhost to listen to **THIS WILL
+VARY DEPENDING UPON WHICH PORT YOU WANT TO LISTEN TO</p></div>
+<div class="paragraph"><p>5000(RIGHT OF localhost)=the port on target host that you want to forward</p></div>
+<div class="paragraph"><p><a href="mailto:root at 10.101.8.50">root at 10.101.8.50</a>=the ip of the Virtual Machine on bionimbus cloud, this will
+vary based on what ip you get for your Virtual Machine instance</p></div>
+<div class="paragraph"><p><tt>coaster-service -port 1984 -localport 35753 -nosec</tt></p></div>
+<div class="paragraph"><p>#On anywhere as long as provide the correct callback uri: here the
+"http://140.221.9.110:42195" is the callback uri of previous ones</p></div>
+<div class="paragraph"><p><tt>worker.pl <a href="http://localhost:42195">http://localhost:42195</a> tmp /tmp # where 42195 is the port where the
+coaster service is listening to the workers</tt></p></div>
</div>
<div class="sect2">
<h3 id="_magellan">3.6. Magellan</h3>
@@ -673,15 +699,16 @@
Include neat diagrams.</p></div>
<div class="sect2">
<h3 id="_for_beginners">4.1. For Beginners</h3>
-<div class="paragraph"><p>Coasters for beginners.</p></div>
+<div class="paragraph"><p>Coasters for beginners. Usage of existing, prebuilt templates.</p></div>
</div>
<div class="sect2">
<h3 id="_for_intermediate_users">4.2. For Intermediate Users</h3>
-<div class="paragraph"><p>Coasters for intermediate users.</p></div>
+<div class="paragraph"><p>Coasters for intermediate users. Usage of gensites to generate your own sites
+configurations.</p></div>
</div>
<div class="sect2">
<h3 id="_for_advanced_users">4.3. For Advanced Users</h3>
-<div class="paragraph"><p>Coasters for advanced users.</p></div>
+<div class="paragraph"><p>Coasters for advanced users. Getting your hands dirty.</p></div>
</div>
<div class="sect2">
<h3 id="_how_to_run_swift_under_different_coasters_configurations">4.4. How to run Swift under different Coasters configurations</h3>
@@ -701,54 +728,16 @@
<div class="sect2">
<h3 id="_summary_of_differences_between_different_coaster_types">4.5. Summary of Differences Between different Coaster types</h3>
<div class="paragraph"><p>A tabular representations of highlights of different coaster setups</p></div>
+<div class="paragraph"><p>Data Management</p></div>
</div>
</div>
</div>
-<div class="sect1">
-<h2 id="_collective_data_management_cdm_and_jets">5. Collective Data Management (CDM) and JETS</h2>
-<div class="sectionbody">
-<div class="paragraph"><p>CDM and JETS</p></div>
-<div class="sect2">
-<h3 id="_cdm">5.1. CDM</h3>
-<div class="paragraph"><p>todo</p></div>
</div>
-<div class="sect2">
-<h3 id="_jets">5.2. JETS</h3>
-<div class="paragraph"><p>todo</p></div>
-</div>
-</div>
-</div>
-<div class="sect1">
-<h2 id="_debugging_swift">6. Debugging Swift</h2>
-<div class="sectionbody">
-<div class="paragraph"><p>De-bug.</p></div>
-<div class="sect2">
-<h3 id="_debugging_heuristics_or_where_to_look_when_things_break">6.1. Debugging Heuristics or Where to look when things break</h3>
-<div class="paragraph"><p>Steps to follow to debug Swift</p></div>
-</div>
-<div class="sect2">
-<h3 id="_swift_logging_and_log4j_setup">6.2. Swift Logging and log4j Setup</h3>
-<div class="paragraph"><p>Logging levels for Swift.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_what_if_swift_hangs">6.3. What if Swift hangs?</h3>
-<div class="paragraph"><p>Swift has a built-in hang-checker that chips in when Swift hangs.</p></div>
-</div>
-</div>
-</div>
-<div class="sect1">
-<h2 id="_deprecated_things_of_swift">7. Deprecated things of Swift</h2>
-<div class="sectionbody">
-<div class="paragraph"><p>Notes on things that have been deprecated and may not be supported in the near
-future.</p></div>
-</div>
-</div>
-</div>
<div id="footnotes"><hr /></div>
<div id="footer">
<div id="footer-text">
Version 0.92<br />
-Last updated 2011-04-13 11:07:15 CDT
+Last updated 2011-04-13 17:54:29 CDT
</div>
</div>
</body>
Modified: www/cookbook/cookbook-asciidoc.txt
===================================================================
--- www/cookbook/cookbook-asciidoc.txt 2011-04-13 20:41:48 UTC (rev 4372)
+++ www/cookbook/cookbook-asciidoc.txt 2011-04-13 22:55:14 UTC (rev 4373)
@@ -16,8 +16,7 @@
Swift programs (or workflows) are written in a language called SwiftScript.
-SwiftScript programs are dataflow oriented - they are primarily concerned with processing (possibly large) collections of data files, by invoking programs to do that processing. Swift handles execution of
-such programs on remote sites by choosing sites, handling the staging of input and output files to and from the chosen sites and remote execution of program code.
+SwiftScript programs are dataflow oriented - they are primarily concerned with processing (possibly large) collections of data files, by invoking programs to do that processing. Swift handles execution of such programs on remote sites by choosing sites, handling the staging of input and output files to and from the chosen sites and remote execution of program code.
Swift Basics
------------
@@ -34,13 +33,21 @@
The environment will be different when using Swift from prebuilt
distribution and trunk.
+Setting transformation catalog
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+tc
+
+Setting swift configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+cf
+
First SwiftScript
~~~~~~~~~~~~~~~~~
Your first SwiftScript
Hello Swift-World!
-Second SwiftScript
+second SwiftScript
~~~~~~~~~~~~~~~~~~~
Your second SwiftScript
catsn
@@ -88,7 +95,36 @@
Bionimbus
~~~~~~~~~
Swift on Bionimbus
+.VERY IMPORTANT TO READ THIS
+CONTAINS INFORMATION ON HOW TO CONNECT BACK TO GATEWAY FROM VIRTUAL MACHINES
+USING REVERSE SSH TUNNELING
+
+TO OPEN A REVERSE SSH TUNNEL DO THE FOLLOWING:
+.FROM THE GATEWAY PROMPT
+
++ssh -R *:5000:localhost:5000 root at 10.101.8.50 sleep 999+
+
+WHERE:
+*=network interface, should remain the same on all cases
+localhost=the gateway host, should remain the same
+
+5000(LEFT OF localhost)=the port number on localhost to listen to **THIS WILL
+VARY DEPENDING UPON WHICH PORT YOU WANT TO LISTEN TO
+
+5000(RIGHT OF localhost)=the port on target host that you want to forward
+
+root at 10.101.8.50=the ip of the Virtual Machine on bionimbus cloud, this will
+vary based on what ip you get for your Virtual Machine instance
+
++coaster-service -port 1984 -localport 35753 -nosec+
+
+#On anywhere as long as provide the correct callback uri: here the
+"http://140.221.9.110:42195" is the callback uri of previous ones
+
++worker.pl http://localhost:42195 tmp /tmp # where 42195 is the port where the
+coaster service is listening to the workers+
+
Magellan
~~~~~~~~
Swift on Magellan
@@ -101,15 +137,16 @@
For Beginners
~~~~~~~~~~~~~~
-Coasters for beginners.
+Coasters for beginners. Usage of existing, prebuilt templates.
For Intermediate Users
~~~~~~~~~~~~~~~~~~~~~~~
-Coasters for intermediate users.
+Coasters for intermediate users. Usage of gensites to generate your own sites
+configurations.
For Advanced Users
~~~~~~~~~~~~~~~~~~~
-Coasters for advanced users.
+Coasters for advanced users. Getting your hands dirty.
How to run Swift under different Coasters configurations
@@ -132,41 +169,5 @@
A tabular representations of highlights of different coaster setups
-Collective Data Management (CDM) and JETS
---------------------------------------------
-CDM and JETS
+Data Management
-CDM
-~~~~
-
-todo
-
-JETS
-~~~~~
-
-todo
-
-Debugging Swift
----------------
-
-De-bug.
-
-Debugging Heuristics or Where to look when things break
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Steps to follow to debug Swift
-
-Swift Logging and log4j Setup
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Logging levels for Swift.
-
-What if Swift hangs?
-~~~~~~~~~~~~~~~~~~~~~
-
-Swift has a built-in hang-checker that chips in when Swift hangs.
-
-
-Deprecated things of Swift
----------------------------
-Notes on things that have been deprecated and may not be supported in the near
-future.
-
More information about the Swift-commit
mailing list