[Swift-commit] r6773 - SwiftTutorials/ATPESC_2013-08-06

[wozniak at ci.uchicago.edu]

Author: wozniak
Date: 2013-08-05 17:13:58 -0500 (Mon, 05 Aug 2013)
New Revision: 6773

Modified:
   SwiftTutorials/ATPESC_2013-08-06/README
Log:
More Swift/T notes


Modified: SwiftTutorials/ATPESC_2013-08-06/README
===================================================================
--- SwiftTutorials/ATPESC_2013-08-06/README	2013-08-05 21:47:52 UTC (rev 6772)
+++ SwiftTutorials/ATPESC_2013-08-06/README	2013-08-05 22:13:58 UTC (rev 6773)
@@ -516,21 +516,66 @@
 include::part11-swift-py-r/code/run-dets.sh.txt[]
 ----
 
+As a reference, an equivalent plain Python code is provided:
+
+.dets.py
+----
+include::part11-swift-py-r/code/dets.py
+----
+
+The Swift script is:
+
+.dets.swift
+----
+include::part11-swift-py-r/code/dets.swift
+----
+
+=== Analysis
+
+The Turbine run time script creates a +TURBINE_OUTPUT+ directory and
+reports it.  Standard output from the job is in +output.txt+.
+
+Use +grep dets+ to find the results printed by the Swift script.  You
+will see each determinant as inserted into the Swift array, and the
+maximal result.
+
+Turbine was launched with +-n 10+; that is, 10 total processes, 8 of
+which are workers, ranks 1-8.  Much of the output is created by rank
+0, the Turbine engine.  (Only one engine is used in this case,
+although Swift/T supports multiple engines for scalability.)  The ADLB
+server runs on rank 9 and produces almost no output.  (Swift/T and
+ADLB support multiple servers.)
+
+You may +grep+ for +python: expression:+ to see the Python expressions
+evaluated on the workers.  Note the rank numbers.
+
+For production cases, you may disable logging by setting
++TURBINE_LOG=0+ in the environment.
+
+=== More information
+
+For more information about Swift/T, see:
+
+* https://sites.google.com/site/exmcomputing/swift-t[Swift/T Overview]
+* http://www.mcs.anl.gov/exm/local/guides/swift.html[Swift/T Guide]
+* http://www.mcs.anl.gov/exm/local/guides/turbine-sites.html[Sites Guide]
+   - notes for running Swift/T on various systems
+
 Running MPI apps under Swift
 ----------------------------
 
 Modis - Satellite image data processing
 ---------------------------------------
 
-In this section we will use swift to process data from a large dataset of 
-files that categorize the Earth's surface, derived from the MODIS sensor 
+In this section we will use swift to process data from a large dataset of
+files that categorize the Earth's surface, derived from the MODIS sensor
 instruments that orbit the Earth on two NASA satellites of the Earth Observing System.
 
-The dataset we use (for 2002, named +mcd12q1+) consists of 317 "tile" files 
+The dataset we use (for 2002, named +mcd12q1+) consists of 317 "tile" files
 that categorize every 250-meter square of non-ocean surface of the Earth into
 one of 17 "land cover" categories (for example, water, ice, forest, barren, urban).
-Each pixel of these data files has a value of 0 to 16, describing one square 
-of the Earth's surface at a specific point in time. Each tile file has 
+Each pixel of these data files has a value of 0 to 16, describing one square
+of the Earth's surface at a specific point in time. Each tile file has
 approximately 5 million 1-byte pixels (5.7 MB), covering 2400x2400 250-meter
 squares, based on a specific map projection.
 
@@ -618,7 +663,7 @@
 files contain information about the top 10 urban areas.
 
 In the previous example, you have noticed that running all 317 input files on
-your laptop, even with 4 tasks a time, is not very efficient. 
+your laptop, even with 4 tasks a time, is not very efficient.
 
 TODO : In the next example, instead of running locally, we will use a cluster called midway at the University of Chicago to improve performance.
 
@@ -739,7 +784,7 @@
 modis05 - Create multi-color tile images
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The fifth modis example extends the previous examples by adding the colorModis app 
+The fifth modis example extends the previous examples by adding the colorModis app
 to create multi-color images for all tiles.
 
 .modis05.swift