[Swift-commit] r6806 - SwiftApps/Scattering/paintgrid/docs
davidk at ci.uchicago.edu
davidk at ci.uchicago.edu
Thu Aug 8 14:12:14 CDT 2013
Author: davidk
Date: 2013-08-08 14:12:14 -0500 (Thu, 08 Aug 2013)
New Revision: 6806
Modified:
SwiftApps/Scattering/paintgrid/docs/README
SwiftApps/Scattering/paintgrid/docs/tutorial.html
Log:
Section on python applications
Modified: SwiftApps/Scattering/paintgrid/docs/README
===================================================================
--- SwiftApps/Scattering/paintgrid/docs/README 2013-08-08 18:22:27 UTC (rev 6805)
+++ SwiftApps/Scattering/paintgrid/docs/README 2013-08-08 19:12:14 UTC (rev 6806)
@@ -6,13 +6,16 @@
Check out scripts from SVN
~~~~~~~~~~~~~~~~~~~~~~~~~~
-To checkout the most recent PaintGrid tutorial scripts from SVN, run the following command:
+To checkout the most recent PaintGrid tutorial scripts from SVN, run the
+following command:
-----
$ svn co https://svn.ci.uchicago.edu/svn/vdl2/SwiftApps/Scattering/paintgrid
-----
-This will create a directory called paintgrid which contains all of the scripts mentioned in this document.
+This will create a directory called paintgrid which contains all of the scripts
+mentioned in this document.
+
Run setup
~~~~~~~~~
Once the scripts are checked out, run the following commands to perform
@@ -23,44 +26,92 @@
$ source setup.sh # Updates PATH and creates a configuration file in $HOME/.swift
-----
-NOTE: If you disconnect from the machine, you will need to re-run source setup.sh.
+The setup.sh script will also create two data files that will be used for later
+processing: data.0001.tiny (800 bytes), and data.0001.tif (8 megabytes).
+NOTE: If you disconnect from the machine, you will need to re-run source
+setup.sh.
-Run the Swift script:
+Mock PaintGrid applications
+---------------------------
+There are two python applications that serve as standins for the real science
+applications: genpoints.py and processpoints.py.
- $ mkdir ~/swift
- $ cp swift.properties ~/.swift
+genpoints.py
+~~~~~~~~~~~~
+The genpoints.py script generates a series of point values based on an input
+parameter file. The command line arguments are:
- $ swift -tc.file apps -sites.file amd.xml paintgrid.swift -runDir=$PWD -params=genpoints.params -image=data.0001.tiny
+.genpoints.py arguments
+[options="header"]
+|======================
+|Argument number|Description
+|1 |Name of parameter file to read
+|2 |Run directory - location to create output files
+|======================
+The input parameter file determines how point value files get created. Below
+is a table with a list of parameters and a brief description.
+.genpoints.py parameter file
+[options="header"]
+|===========================
+|Parameter|Description
+|minx|Minimum x value
+|maxx|Maximum x value
+|miny|Minimum y value
+|maxy|Maximum y value
+|minz|Minimum x value
+|maxz|Maximum z value
+|incr|Increment size
+|tuplesPerFile|Number of tuples per file
+|filePrefix|Prefix of point value files
+|outDir|Output directory name, relative to run directory
+|==========================
-TODO:
+Below is an example genpoints.py parameter file called onepoint.params that with max values
+at 10 and increments at 10, causing the generation of only one point file
+-----
+minx=0.0
+maxx=10.0
+miny=0.0
+maxy=10.0
+minz=0.0
+maxz=10.0
+incr=10.0
+-----
-Execution scenarios:
+Here is an example putting everything together and viewing the output:
+-----
+$ ./genpoints.py onepoint.params /clhome/DAVIDK/Scattering/paintgrid
+out/seq/seq00000
+$ cat out/seq/seq00000
+0.0 0.0 0.0
+-----
- run.local.sh
+processpoints.py
+~~~~~~~~~~~~~~~~
+The processpoints.py script simulates an image processing application. It reads
+a point file generated by genpoints.py, counts the number of bytes in a given
+data file, and then sleeps for a requested amount of time.
- run.beagle.sh
+.processpoints.py arguments
+[options="header"]
+|==========================
+|Parameter|Description
+|1 | Name of image file to read
+|2 | Name of point file to read
+|3 | Time to sleep (floating point seconds)
+|==========================
- run.midway.sh
+Here is an example of manually running processpoints.py:
+-----
+$ ./processpoints.py data.0001.tiny out/seq/seq00000 1
+Data file has 800 pixels
+Processing 1 points in model space
+Runtime is 1.0 seconds per point
+ 0.000000 0.000000 0.000000 0.000000
+-----
- run.midway+beagle.sh
- run.blues.sh
-
- run.orthros.sh
-
-
-Data transfer models:
-
-- provider staging without caching
-
-- provider staging with ad-hoc caching of big common files
-
-- gridftp staging
-
-- ssh (scp) staging
-
-- wrapper staging with scp and caching
-
+swift -tc.file apps -sites.file amd.xml paintgrid.swift -runDir=$PWD -params=genpoints.params -image=data.0001.tiny
Modified: SwiftApps/Scattering/paintgrid/docs/tutorial.html
===================================================================
--- SwiftApps/Scattering/paintgrid/docs/tutorial.html 2013-08-08 18:22:27 UTC (rev 6805)
+++ SwiftApps/Scattering/paintgrid/docs/tutorial.html 2013-08-08 19:12:14 UTC (rev 6806)
@@ -581,12 +581,14 @@
<div class="sectionbody">
<div class="sect2">
<h3 id="_check_out_scripts_from_svn">Check out scripts from SVN</h3>
-<div class="paragraph"><p>To checkout the most recent PaintGrid tutorial scripts from SVN, run the following command:</p></div>
+<div class="paragraph"><p>To checkout the most recent PaintGrid tutorial scripts from SVN, run the
+following command:</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>$ svn co https://svn.ci.uchicago.edu/svn/vdl2/SwiftApps/Scattering/paintgrid</tt></pre>
</div></div>
-<div class="paragraph"><p>This will create a directory called paintgrid which contains all of the scripts mentioned in this document.</p></div>
+<div class="paragraph"><p>This will create a directory called paintgrid which contains all of the scripts
+mentioned in this document.</p></div>
</div>
<div class="sect2">
<h3 id="_run_setup">Run setup</h3>
@@ -597,78 +599,181 @@
<pre><tt>$ cd paintgrid # change to the newly created paintgrid directory
$ source setup.sh # Updates PATH and creates a configuration file in $HOME/.swift</tt></pre>
</div></div>
+<div class="paragraph"><p>The setup.sh script will also create two data files that will be used for later
+processing: data.0001.tiny (800 bytes), and data.0001.tif (8 megabytes).</p></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<div class="title">Note</div>
</td>
-<td class="content">If you disconnect from the machine, you will need to re-run source setup.sh.</td>
+<td class="content">If you disconnect from the machine, you will need to re-run source
+setup.sh.</td>
</tr></table>
</div>
-<div class="paragraph"><p>Run the Swift script:</p></div>
-<div class="literalblock">
+</div>
+</div>
+</div>
+<div class="sect1">
+<h2 id="_mock_paintgrid_applications">Mock PaintGrid applications</h2>
+<div class="sectionbody">
+<div class="paragraph"><p>There are two python applications that serve as standins for the real science
+applications: genpoints.py and processpoints.py.</p></div>
+<div class="sect2">
+<h3 id="_genpoints_py">genpoints.py</h3>
+<div class="paragraph"><p>The genpoints.py script generates a series of point values based on an input
+parameter file. The command line arguments are:</p></div>
+<div class="tableblock">
+<table rules="all"
+width="100%"
+frame="border"
+cellspacing="0" cellpadding="4">
+<caption class="title">Table 1. genpoints.py arguments</caption>
+<col width="50%" />
+<col width="50%" />
+<thead>
+<tr>
+<th align="left" valign="top">Argument number</th>
+<th align="left" valign="top">Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td align="left" valign="top"><p class="table">1</p></td>
+<td align="left" valign="top"><p class="table">Name of parameter file to read</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">2</p></td>
+<td align="left" valign="top"><p class="table">Run directory - location to create output files</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+<div class="paragraph"><p>The input parameter file determines how point value files get created. Below
+is a table with a list of parameters and a brief description.</p></div>
+<div class="tableblock">
+<table rules="all"
+width="100%"
+frame="border"
+cellspacing="0" cellpadding="4">
+<caption class="title">Table 2. genpoints.py parameter file</caption>
+<col width="50%" />
+<col width="50%" />
+<thead>
+<tr>
+<th align="left" valign="top">Parameter</th>
+<th align="left" valign="top">Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td align="left" valign="top"><p class="table">minx</p></td>
+<td align="left" valign="top"><p class="table">Minimum x value</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">maxx</p></td>
+<td align="left" valign="top"><p class="table">Maximum x value</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">miny</p></td>
+<td align="left" valign="top"><p class="table">Minimum y value</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">maxy</p></td>
+<td align="left" valign="top"><p class="table">Maximum y value</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">minz</p></td>
+<td align="left" valign="top"><p class="table">Minimum x value</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">maxz</p></td>
+<td align="left" valign="top"><p class="table">Maximum z value</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">incr</p></td>
+<td align="left" valign="top"><p class="table">Increment size</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">tuplesPerFile</p></td>
+<td align="left" valign="top"><p class="table">Number of tuples per file</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">filePrefix</p></td>
+<td align="left" valign="top"><p class="table">Prefix of point value files</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">outDir</p></td>
+<td align="left" valign="top"><p class="table">Output directory name, relative to run directory</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+<div class="paragraph"><p>Below is an example genpoints.py parameter file called onepoint.params that with max values
+at 10 and increments at 10, causing the generation of only one point file</p></div>
+<div class="listingblock">
<div class="content">
-<pre><tt>$ mkdir ~/swift
-$ cp swift.properties ~/.swift</tt></pre>
+<pre><tt>minx=0.0
+maxx=10.0
+miny=0.0
+maxy=10.0
+minz=0.0
+maxz=10.0
+incr=10.0</tt></pre>
</div></div>
-<div class="literalblock">
+<div class="paragraph"><p>Here is an example putting everything together and viewing the output:</p></div>
+<div class="listingblock">
<div class="content">
-<pre><tt>$ swift -tc.file apps -sites.file amd.xml paintgrid.swift -runDir=$PWD -params=genpoints.params -image=data.0001.tiny</tt></pre>
+<pre><tt>$ ./genpoints.py onepoint.params /clhome/DAVIDK/Scattering/paintgrid
+out/seq/seq00000
+$ cat out/seq/seq00000
+0.0 0.0 0.0</tt></pre>
</div></div>
-<div class="paragraph"><p>TODO:</p></div>
-<div class="paragraph"><p>Execution scenarios:</p></div>
-<div class="literalblock">
+</div>
+<div class="sect2">
+<h3 id="_processpoints_py">processpoints.py</h3>
+<div class="paragraph"><p>The processpoints.py script simulates an image processing application. It reads
+a point file generated by genpoints.py, counts the number of bytes in a given
+data file, and then sleeps for a requested amount of time.</p></div>
+<div class="tableblock">
+<table rules="all"
+width="100%"
+frame="border"
+cellspacing="0" cellpadding="4">
+<caption class="title">Table 3. processpoints.py arguments</caption>
+<col width="50%" />
+<col width="50%" />
+<thead>
+<tr>
+<th align="left" valign="top">Parameter</th>
+<th align="left" valign="top">Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td align="left" valign="top"><p class="table">1</p></td>
+<td align="left" valign="top"><p class="table">Name of image file to read</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">2</p></td>
+<td align="left" valign="top"><p class="table">Name of point file to read</p></td>
+</tr>
+<tr>
+<td align="left" valign="top"><p class="table">3</p></td>
+<td align="left" valign="top"><p class="table">Time to sleep (floating point seconds)</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+<div class="paragraph"><p>Here is an example of manually running processpoints.py:</p></div>
+<div class="listingblock">
<div class="content">
-<pre><tt>run.local.sh</tt></pre>
+<pre><tt>$ ./processpoints.py data.0001.tiny out/seq/seq00000 1
+Data file has 800 pixels
+Processing 1 points in model space
+Runtime is 1.0 seconds per point
+ 0.000000 0.000000 0.000000 0.000000</tt></pre>
</div></div>
-<div class="literalblock">
-<div class="content">
-<pre><tt>run.beagle.sh</tt></pre>
-</div></div>
-<div class="literalblock">
-<div class="content">
-<pre><tt>run.midway.sh</tt></pre>
-</div></div>
-<div class="literalblock">
-<div class="content">
-<pre><tt>run.midway+beagle.sh</tt></pre>
-</div></div>
-<div class="literalblock">
-<div class="content">
-<pre><tt>run.blues.sh</tt></pre>
-</div></div>
-<div class="literalblock">
-<div class="content">
-<pre><tt>run.orthros.sh</tt></pre>
-</div></div>
-<div class="paragraph"><p>Data transfer models:</p></div>
-<div class="ulist"><ul>
-<li>
-<p>
-provider staging without caching
-</p>
-</li>
-<li>
-<p>
-provider staging with ad-hoc caching of big common files
-</p>
-</li>
-<li>
-<p>
-gridftp staging
-</p>
-</li>
-<li>
-<p>
-ssh (scp) staging
-</p>
-</li>
-<li>
-<p>
-wrapper staging with scp and caching
-</p>
-</li>
-</ul></div>
+<div class="paragraph"><p>swift -tc.file apps -sites.file amd.xml paintgrid.swift -runDir=$PWD -params=genpoints.params -image=data.0001.tiny</p></div>
</div>
</div>
</div>
@@ -676,7 +781,7 @@
<div id="footnotes"><hr /></div>
<div id="footer">
<div id="footer-text">
-Last updated 2013-08-08 13:12:10 CDT
+Last updated 2013-08-08 14:07:42 CDT
</div>
</div>
</body>
More information about the Swift-commit
mailing list