[Swift-commit] r7397 - SwiftApps/Swift-MapRed/mapred_combiner_V4

[ketan at ci.uchicago.edu]

Author: ketan
Date: 2013-12-10 14:41:14 -0600 (Tue, 10 Dec 2013)
New Revision: 7397

Modified:
   SwiftApps/Swift-MapRed/mapred_combiner_V4/README
   SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper_template.sh
Log:
-

Modified: SwiftApps/Swift-MapRed/mapred_combiner_V4/README
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/README	2013-12-10 20:25:35 UTC (rev 7396)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/README	2013-12-10 20:41:14 UTC (rev 7397)
@@ -4,13 +4,15 @@
 REQUIREMENTS:
 -------------
 
-1. CCTools - Install cctools on every site.
+1. CCTools - Install cctools on each site.
    http://www3.nd.edu/~ccl/software/manuals/install.html
 
-Run MapReduce style applications with Swift. Since swift is a file based
+Note: On Midway cluster, CCTools are preinstalled on Yadu's home.
+
+Run MapReduce style applications with Swift. Since Swift is a file based
 model as opposed to the Key-Value record based approach of MapReduce, there
 are differences in the way the user supplied Map and Reduce functions
-are required to work.
+work.
 
 setup.sh
 --------
@@ -18,19 +20,19 @@
 setup.sh also compiles and copies the exec server and client to the remote sites.
 
 Map function
----------------
+-------------
 The user supplied Map function could be any executable that accepts the
-arguments defined in the swift script and should return its result to stdout. #fixme
-You can use any number of arguments/files in the swift script and expect them
+arguments defined in the Swift script and should return its result to stdout. #fixme
+You can use any number of arguments/files in the Swift script and expect them
 to be passed transparently to your map application.
 
-Note: Only results sent to stdout from map will be accepted by the Swiftreduce
+Note: Only results sent to stdout from map will be accepted by the SwiftReduce
 framework.
 
 Reduce function
-------------------
+----------------
 The reduce function can by any executable that accepts one or more files
-from the Map function. Results should be sent to stdout and Swiftreduce
+from the Map function. Results should be sent to stdout and SwiftReduce
 will route them. If the application is distributive (commutative and associative)
 you may use different reduce styles, such as local combine, k-way reduction tree.
 
@@ -54,13 +56,13 @@
 Each record consists of 1 random number and count, padded to 100 bytes.
 So, in the usual run a file of 10^8 records generated would have 10^6 bytes.
 
-Example - Gigaasort
+Example - Gigasort
 ===================
 Map - Generates 100 byte records containing 1 random number.
 Reduce - Accepts one or more files, sorts each file and merge sorts the
     sorted files.
 
-Example - Blastreduce
+Example - BlastReduce
 =====================
 Run the blast reduce example as follows:
 In setup.sh change the BEAGLE_USERNAME and MIDWAY_USERNAME to your user name on Beagle and Midway respectively.
@@ -73,7 +75,6 @@
 
 Describe workflow
 =================
-
 teragen will split the task of generating files containing 10^8 records (~100 MB)
 over 10000 tasks, thus generating in total 1 Tera records.
 

Modified: SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper_template.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper_template.sh	2013-12-10 20:25:35 UTC (rev 7396)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper_template.sh	2013-12-10 20:41:14 UTC (rev 7397)
@@ -1,7 +1,8 @@
 #!/bin/bash
 
-# By default with ARG1:100 and SLICESIZE=10000, this script will generate
+# By default with ARG1:=100 and SLICESIZE=10000, this script will generate
 # 10^6 records.
+
 ARG1=1
 [ ! -z $1 ] && ARG1=$1