[Swift-commit] r7336 - SwiftApps/Swift-MapRed/mapred_combiner_V4
ketan at ci.uchicago.edu
ketan at ci.uchicago.edu
Tue Nov 26 11:29:13 CST 2013
Author: ketan
Date: 2013-11-26 11:29:13 -0600 (Tue, 26 Nov 2013)
New Revision: 7336
Modified:
SwiftApps/Swift-MapRed/mapred_combiner_V4/README
Log:
update readme
Modified: SwiftApps/Swift-MapRed/mapred_combiner_V4/README
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/README 2013-11-26 17:07:01 UTC (rev 7335)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/README 2013-11-26 17:29:13 UTC (rev 7336)
@@ -2,10 +2,11 @@
============
REQUIREMENTS:
+-------------
+
1. CCTools - Install cctools on every site.
http://www3.nd.edu/~ccl/software/manuals/install.html
-
Run MapReduce style applications with Swift. Since swift is a file based
model as opposed to the Key-Value record based approach of MapReduce, there
are differences in the way the user supplied Map and Reduce functions
@@ -13,25 +14,23 @@
setup.sh
--------
+Modify the USERNAME variables in setup.sh to configure scripts appropriately.
+setup.sh also compiles and copies the exec server and client to the remote sites.
-Modify the USERNAME variables in setup.sh to condifure scripts appropriately
-Setup.sh also compiles are copies the exec server and client to the remote sites.
-
-Map application
+Map function
---------------
-The user supplied Map application could be any executable that accepts the
-arguments defined in the swift script and should return its result to stdout.
-You can use any number of arguments/files in the swiftscript and expect them
+The user supplied Map function could be any executable that accepts the
+arguments defined in the swift script and should return its result to stdout. #fixme
+You can use any number of arguments/files in the swift script and expect them
to be passed transparently to your map application.
Note: Only results sent to stdout from map will be accepted by the Swiftreduce
framework.
-Reduce application
+Reduce function
------------------
-
-The reduce application can by any executable that accepts one or more files
-from the Map application. Results should be sent to stdout and Swiftreduce
+The reduce function can by any executable that accepts one or more files
+from the Map function. Results should be sent to stdout and Swiftreduce
will route them. If the application is distributive (commutative and associative)
you may use different reduce styles, such as local combine, k-way reduction tree.
@@ -40,7 +39,6 @@
run.sh
------
-
USAGE: ./run.sh <sites.xml> <no: of map jobs> <args>
In the default example ./run.sh multiple.xml 10 10
@@ -64,7 +62,6 @@
Example - Blastreduce
=====================
-
Run the blast reduce example as follows:
swift -sites.file sites.local.xml -tc.file tc.data -config swift.properties blastreduce.swift
@@ -81,4 +78,3 @@
TODOs and Future Work
=====================
1. Builds for exec_{server,client} on different architectures.
-2.
More information about the Swift-commit
mailing list