[Swift-commit] r4379 - www/cookbook
ketan at ci.uchicago.edu
ketan at ci.uchicago.edu
Fri Apr 15 19:20:59 CDT 2011
Author: ketan
Date: 2011-04-15 19:20:59 -0500 (Fri, 15 Apr 2011)
New Revision: 4379
Modified:
www/cookbook/cookbook-asciidoc.html
www/cookbook/cookbook-asciidoc.txt
Log:
some refinements
Modified: www/cookbook/cookbook-asciidoc.html
===================================================================
--- www/cookbook/cookbook-asciidoc.html 2011-04-15 22:03:33 UTC (rev 4378)
+++ www/cookbook/cookbook-asciidoc.html 2011-04-16 00:20:59 UTC (rev 4379)
@@ -923,8 +923,29 @@
<div class="paragraph"><p><tt>swift -resume ftdock-[id].0.rlog \[rest of the exact command line from initial
run\]</tt></p></div>
</div>
+<div class="sect2">
+<h3 id="_passing_an_array_to_swift">2.7. Passing an array to swift?</h3>
+<div class="olist arabic"><ol class="arabic">
+<li>
+<p>
+you can write the array to a file and read in in swift using
+readData (or readData2).
+</p>
+</li>
+<li>
+<p>
+direct the array into a file (possibly with a "here document" which expands the array) and then read the file in Swift with readData() or process it with a Swift app() function?
+</p>
+</li>
+<li>
+<p>
+you can also use @strsplit on a comma separated command line arg and that works well for me.
+</p>
+</li>
+</ol></div>
</div>
</div>
+</div>
<div class="sect1">
<h2 id="_swift_on_8230">3. Swift on …</h2>
<div class="sectionbody">
@@ -946,19 +967,7 @@
will get fast prioritized execution. Good for small tests</p></div>
</div>
<div class="sect2">
-<h3 id="_intrepid_bg_p">3.2. Intrepid-BG/P</h3>
-<div class="paragraph"><p>Swift on Intrepid-BG/P</p></div>
-</div>
-<div class="sect2">
-<h3 id="_pads">3.3. PADS</h3>
-<div class="paragraph"><p>Swift on PADS</p></div>
-</div>
-<div class="sect2">
-<h3 id="_osg">3.4. OSG</h3>
-<div class="paragraph"><p>Swift on OSG</p></div>
-</div>
-<div class="sect2">
-<h3 id="_bionimbus">3.5. Bionimbus</h3>
+<h3 id="_bionimbus">3.2. Bionimbus</h3>
<div class="paragraph"><p>Swift on Bionimbus
.very important to read this</p></div>
<div class="paragraph"><p>contains information on how to connect back to gateway from virtual machines
@@ -980,23 +989,16 @@
<div class="paragraph"><p><tt>worker.pl <a href="http://localhost:42195">http://localhost:42195</a> tmp /tmp # where 42195 is the port where the
coaster service is listening to the workers</tt></p></div>
</div>
-<div class="sect2">
-<h3 id="_magellan">3.6. Magellan</h3>
-<div class="paragraph"><p>Swift on Magellan</p></div>
</div>
</div>
-</div>
<div class="sect1">
<h2 id="_coasters">4. Coasters</h2>
<div class="sectionbody">
-<div class="paragraph"><p>Describe coasters mechanisms.
-<strong>Include neat diagrams.</strong></p></div>
-<div class="paragraph"><p>A nice coasters setup case-study:</p></div>
+<div class="paragraph"><p>Describe coasters mechanisms.</p></div>
+<div class="paragraph"><p>Following is a nice coasters setup case-study where sites.xml coaster settings were:</p></div>
<div class="listingblock">
<div class="content">
-<pre><tt>Your main sites.xml coaster settings were:
-
-<execution provider="coaster" jobmanager="local:pbs"/>
+<pre><tt><execution provider="coaster" jobmanager="local:pbs"/>
<profile namespace="globus" key="project">CI-CCR000013</profile>
<profile namespace="globus" key="ppn">24:cray:pack</profile>
<profile namespace="globus" key="workersPerNode">24</profile>
@@ -1062,65 +1064,12 @@
answer to make usage simpler.</tt></pre>
</div></div>
<div class="sect2">
-<h3 id="_for_beginners">4.1. For Beginners</h3>
-<div class="paragraph"><p>Coasters for beginners. Usage of existing, prebuilt templates.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_for_intermediate_users">4.2. For Intermediate Users</h3>
-<div class="paragraph"><p>Coasters for intermediate users.</p></div>
-<div class="sect3">
-<h4 id="_using_gensites">4.2.1. Using gensites</h4>
-<div class="paragraph"><p>Usage of gensites to generate your own sites
-configurations.</p></div>
-</div>
-</div>
-<div class="sect2">
-<h3 id="_for_advanced_users">4.3. For Advanced Users</h3>
-<div class="paragraph"><p>Coasters for advanced users. Getting your hands dirty.</p></div>
-</div>
-<div class="sect2">
-<h3 id="_how_to_run_swift_under_different_coasters_configurations">4.4. How to run Swift under different Coasters configurations</h3>
-<div class="sect3">
-<h4 id="_manual">4.4.1. Manual</h4>
-<div class="paragraph"><p>Todo</p></div>
-</div>
-<div class="sect3">
-<h4 id="_passive">4.4.2. Passive</h4>
-<div class="paragraph"><p>Todo</p></div>
-</div>
-<div class="sect3">
-<h4 id="_persistent">4.4.3. Persistent</h4>
-<div class="paragraph"><p>Todo</p></div>
-</div>
-</div>
-<div class="sect2">
-<h3 id="_summary_of_differences_between_different_coaster_types">4.5. Summary of Differences Between different Coaster types</h3>
-<div class="paragraph"><p>A tabular representations of highlights of different coaster setups</p></div>
-<div class="paragraph"><p>Data Management</p></div>
-</div>
-<div class="sect2">
-<h3 id="_debugging_swift">4.6. Debugging Swift</h3>
-<div class="paragraph"><p>Note: The Swift installation at /home/wilde/bigdata/swift includes a
-swift.properties file that has been modified to over-ride the defaults with
-the following property values that are useful for script debugging:</p></div>
-<div class="paragraph"><p>execution.retries=0</p></div>
-<div class="paragraph"><p>sitedir.keep=true</p></div>
-<div class="paragraph"><p>status.mode=provider</p></div>
-<div class="paragraph"><p>wrapperlog.always.transfer=true</p></div>
-<div class="admonitionblock">
-<table><tr>
-<td class="icon">
-<div class="title">Note</div>
-</td>
-<td class="content">How to clone copies of swift.properties and things to watch for
-(like the macros at the start, and unexpected properties in ~/.swift.</td>
-</tr></table>
-</div>
+<h3 id="_debugging_swift">4.1. Debugging Swift</h3>
<div class="paragraph"><p>Swift errors are logged in several places:</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
-all text from standard output and standard error produced by running the
+All text from standard output and standard error produced by running the
swift command
</p>
</li>
@@ -1153,24 +1102,62 @@
</ol></div>
</div>
<div class="sect2">
-<h3 id="_problem_reporting">4.7. Problem Reporting</h3>
+<h3 id="_problem_reporting">4.2. Problem Reporting</h3>
<div class="paragraph"><p>When reporting problems to <a href="mailto:swift-user at ci.uchicago.edu">swift-user at ci.uchicago.edu</a>, please attach the
following files and information:</p></div>
-<div class="paragraph"><p>tc.data and sites.xml (or whatever you named these files)
+<div class="olist arabic"><ol class="arabic">
+<li>
+<p>
+tc.data and sites.xml (or whatever you named these files)
+</p>
+</li>
+<li>
+<p>
your .swift source file and any .swift files it imports
+</p>
+</li>
+<li>
+<p>
any external mapper scripts called by your .swift script
+</p>
+</li>
+<li>
+<p>
all text from standard output and standard error produced by running the
swift command
+</p>
+</li>
+<li>
+<p>
The .log file from this run. It will be named swiftscript.uniqueID.log
+</p>
+</li>
+<li>
+<p>
where "swiftscript" is the name of your *.swift script source file, and
uniqueID is a long unique id which starts with the date and time you ran the
swift command.
+</p>
+</li>
+<li>
+<p>
The swift command line you invoked
+</p>
+</li>
+<li>
+<p>
Any swift.properties entries you over-rode ($HOME/.swift/swift.properties,
-config.file argument properties file, any changes to etc/swift.proerties from
your swift distribution)
+</p>
+</li>
+<li>
+<p>
Which swift distribution you are running (release; svn revisions; other
-local changes you mave have made or included)</p></div>
+local changes you mave have made or included)
+</p>
+</li>
+</ol></div>
</div>
</div>
</div>
@@ -1179,7 +1166,7 @@
<div id="footer">
<div id="footer-text">
Version 0.92<br />
-Last updated 2011-04-15 10:21:37 CDT
+Last updated 2011-04-15 19:20:00 CDT
</div>
</div>
</body>
Modified: www/cookbook/cookbook-asciidoc.txt
===================================================================
--- www/cookbook/cookbook-asciidoc.txt 2011-04-15 22:03:33 UTC (rev 4378)
+++ www/cookbook/cookbook-asciidoc.txt 2011-04-16 00:20:59 UTC (rev 4379)
@@ -253,6 +253,7 @@
Also includes a description of Swift inputs and outputs.
+
Resuming a stopped or crashed Swift Run
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -343,7 +344,14 @@
+swift -resume ftdock-[id].0.rlog \[rest of the exact command line from initial
run\]+
+Passing an array to swift?
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+. you can write the array to a file and read in in swift using
+readData (or readData2).
+. direct the array into a file (possibly with a "here document" which expands the array) and then read the file in Swift with readData() or process it with a Swift app() function?
+. you can also use @strsplit on a comma separated command line arg and that works well for me.
+
Swift on ...
------------
@@ -369,19 +377,19 @@
Note: Running from sandbox node or requesting 1 hour walltime for upto 3 nodes
will get fast prioritized execution. Good for small tests
-
-Intrepid-BG/P
-~~~~~~~~~~~~~
-Swift on Intrepid-BG/P
-
-PADS
-~~~~
-Swift on PADS
-
-OSG
-~~~
-Swift on OSG
-
+//
+//Intrepid-BG/P
+//~~~~~~~~~~~~~
+//Swift on Intrepid-BG/P
+//
+//PADS
+//~~~~
+//Swift on PADS
+//
+//OSG
+//~~~
+//Swift on OSG
+//
Bionimbus
~~~~~~~~~
Swift on Bionimbus
@@ -415,19 +423,17 @@
+worker.pl http://localhost:42195 tmp /tmp # where 42195 is the port where the
coaster service is listening to the workers+
-Magellan
-~~~~~~~~
-Swift on Magellan
-
+//Magellan
+//~~~~~~~~
+//Swift on Magellan
+//
Coasters
--------
Describe coasters mechanisms.
-**Include neat diagrams.**
+//**Include neat diagrams.**
-A nice coasters setup case-study:
+Following is a nice coasters setup case-study where sites.xml coaster settings were:
----
-Your main sites.xml coaster settings were:
-
<execution provider="coaster" jobmanager="local:pbs"/>
<profile namespace="globus" key="project">CI-CCR000013</profile>
<profile namespace="globus" key="ppn">24:cray:pack</profile>
@@ -494,69 +500,55 @@
answer to make usage simpler.
----
+//
+//For Beginners
+//~~~~~~~~~~~~~~
+//Coasters for beginners. Usage of existing, prebuilt templates.
+//
+//For Intermediate Users
+//~~~~~~~~~~~~~~~~~~~~~~~
+//Coasters for intermediate users.
+//
+//Using gensites
+//^^^^^^^^^^^^^^^
+//Usage of gensites to generate your own sites
+//configurations.
+//
+//
+//
+//For Advanced Users
+//~~~~~~~~~~~~~~~~~~~
+//Coasters for advanced users. Getting your hands dirty.
+//
+//
+//How to run Swift under different Coasters configurations
+//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+//
+//Manual
+//^^^^^^^
+//Todo
+//
+//Passive
+//^^^^^^^^
+//Todo
+//
+//Persistent
+//^^^^^^^^^^^
+//Todo
+//
+//Summary of Differences Between different Coaster types
+//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+//A tabular representations of highlights of different coaster setups
+//
+//
+//Data Management
+//
-For Beginners
-~~~~~~~~~~~~~~
-Coasters for beginners. Usage of existing, prebuilt templates.
-
-For Intermediate Users
-~~~~~~~~~~~~~~~~~~~~~~~
-Coasters for intermediate users.
-
-Using gensites
-^^^^^^^^^^^^^^^
-Usage of gensites to generate your own sites
-configurations.
-
-
-
-For Advanced Users
-~~~~~~~~~~~~~~~~~~~
-Coasters for advanced users. Getting your hands dirty.
-
-
-How to run Swift under different Coasters configurations
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Manual
-^^^^^^^
-Todo
-
-Passive
-^^^^^^^^
-Todo
-
-Persistent
-^^^^^^^^^^^
-Todo
-
-Summary of Differences Between different Coaster types
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-A tabular representations of highlights of different coaster setups
-
-
-Data Management
-
Debugging Swift
~~~~~~~~~~~~~~~~
-Note: The Swift installation at /home/wilde/bigdata/swift includes a
-swift.properties file that has been modified to over-ride the defaults with
-the following property values that are useful for script debugging:
-
-execution.retries=0
-
-sitedir.keep=true
-
-status.mode=provider
-
-wrapperlog.always.transfer=true
-
-NOTE: How to clone copies of swift.properties and things to watch for
-(like the macros at the start, and unexpected properties in ~/.swift.
-
Swift errors are logged in several places:
-. all text from standard output and standard error produced by running the
+. All text from standard output and standard error produced by running the
swift command
. The .log file from this run. It will be named swiftscript.uniqueID.log
where "swiftscript" is the name of your *.swift script source file, and
@@ -567,26 +559,35 @@
. $HOME/.globus/scripts directory on the host on which you run the Swift
command, when swift is submitting to a local scheduler (Condor, PBS, SGE,
Cobalt)
- . $HOME/.globus/??? on remote systems that you access via Globus
+. $HOME/.globus/??? on remote systems that you access via Globus
Problem Reporting
~~~~~~~~~~~~~~~~~
When reporting problems to swift-user at ci.uchicago.edu, please attach the
following files and information:
+
+. tc.data and sites.xml (or whatever you named these files)
-tc.data and sites.xml (or whatever you named these files)
-your .swift source file and any .swift files it imports
-any external mapper scripts called by your .swift script
-all text from standard output and standard error produced by running the
+. your .swift source file and any .swift files it imports
+
+. any external mapper scripts called by your .swift script
+
+. all text from standard output and standard error produced by running the
swift command
-The .log file from this run. It will be named swiftscript.uniqueID.log
-where "swiftscript" is the name of your *.swift script source file, and
+
+. The .log file from this run. It will be named swiftscript.uniqueID.log
+
+. where "swiftscript" is the name of your *.swift script source file, and
uniqueID is a long unique id which starts with the date and time you ran the
swift command.
-The swift command line you invoked
-Any swift.properties entries you over-rode ($HOME/.swift/swift.properties,
+
+. The swift command line you invoked
+
+. Any swift.properties entries you over-rode ($HOME/.swift/swift.properties,
-config.file argument properties file, any changes to etc/swift.proerties from
your swift distribution)
-Which swift distribution you are running (release; svn revisions; other
+
+. Which swift distribution you are running (release; svn revisions; other
local changes you mave have made or included)
+
More information about the Swift-commit
mailing list