[Swift-commit] r6879 - SwiftApps/Swift-MapRed/mapred_chirp_combiner
yadunandb at ci.uchicago.edu
yadunandb at ci.uchicago.edu
Mon Aug 19 10:37:34 CDT 2013
Author: yadunandb
Date: 2013-08-19 10:37:33 -0500 (Mon, 19 Aug 2013)
New Revision: 6879
Added:
SwiftApps/Swift-MapRed/mapred_chirp_combiner/client.c
SwiftApps/Swift-MapRed/mapred_chirp_combiner/compile
SwiftApps/Swift-MapRed/mapred_chirp_combiner/parcombiner_c_exec.sh
SwiftApps/Swift-MapRed/mapred_chirp_combiner/server.c
SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen_wrap_c_exec.sh
Modified:
SwiftApps/Swift-MapRed/mapred_chirp_combiner/LAB_RECORDS
SwiftApps/Swift-MapRed/mapred_chirp_combiner/RESULT_FILE
SwiftApps/Swift-MapRed/mapred_chirp_combiner/TEMP
SwiftApps/Swift-MapRed/mapred_chirp_combiner/err_file
SwiftApps/Swift-MapRed/mapred_chirp_combiner/final_result
SwiftApps/Swift-MapRed/mapred_chirp_combiner/recombiner.sh
SwiftApps/Swift-MapRed/mapred_chirp_combiner/run.sh
SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen.kml
SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen.swift
Log:
Committing a tiny c program for remote-exec
* tested single shot between two servers in mcs network.
* seems to fail when multiple clients run on same machine,
possibly because ports clash.
Modified: SwiftApps/Swift-MapRed/mapred_chirp_combiner/LAB_RECORDS
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/LAB_RECORDS 2013-08-16 23:22:26 UTC (rev 6878)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/LAB_RECORDS 2013-08-19 15:37:33 UTC (rev 6879)
@@ -837,3 +837,346 @@
ACTUAL COUNT : 10000000
EXPECTED COUNT: 10000000
---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:20 X FILESIZE:50
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0206-x4peu233
+Progress: time: Mon, 19 Aug 2013 02:07:04 -0500
+chunks 20 filesize 50 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 10000000
+chirp_put: couldn't put ./recombiner.sh to host crank: Permission denied
+couldn't get crank:crank.17255.imd: Permission denied
+cat: crank.17255.txt: No such file or directory
+cat: crank.17255.txt: No such file or directory
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:10 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0207-zf1c0z9c
+Progress: time: Mon, 19 Aug 2013 02:07:49 -0500
+Progress: time: Mon, 19 Aug 2013 02:08:05 -0500 Submitting:9 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:08:09 -0500 Submitting:8 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 02:08:14 -0500 Submitting:6 Submitted:1 Finished successfully:3
+Progress: time: Mon, 19 Aug 2013 02:08:19 -0500 Submitting:4 Finished successfully:6
+Progress: time: Mon, 19 Aug 2013 02:08:23 -0500 Submitting:3 Submitted:1 Finished successfully:6
+Progress: time: Mon, 19 Aug 2013 02:08:27 -0500 Submitted:1 Finished successfully:9
+Progress: time: Mon, 19 Aug 2013 02:08:35 -0500 Submitted:1 Finished successfully:10
+Progress: time: Mon, 19 Aug 2013 02:08:36 -0500 Active:1 Finished successfully:10
+Final status: Mon, 19 Aug 2013 02:08:36 -0500 Finished successfully:11
+real 56.20
+user 8.76
+sys 2.05
+chunks 10 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 1000000
+parcombiner_c_exec.sh: line 40: exec_client: command not found
+parcombiner_c_exec.sh: line 40: exec_client: command not found
+parcombiner_c_exec.sh: line 40: exec_client: command not found
+parcombiner_c_exec.sh: line 40: exec_client: command not found
+parcombiner_c_exec.sh: line 40: exec_client: command not found
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0210-k02nzco4
+Progress: time: Mon, 19 Aug 2013 02:10:52 -0500
+Progress: time: Mon, 19 Aug 2013 02:11:04 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:11:08 -0500 Submitting:1 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 02:11:13 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:11:22 -0500 Submitted:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:11:23 -0500 Finished successfully:4
+real 41.88
+user 6.94
+sys 1.22
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+client: connect: Connection refused
+client: failed to connect
+client: connect: Connection refused
+client: failed to connect
+client: connect: Connection refused
+client: failed to connect
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0215-22x2irid
+Progress: time: Mon, 19 Aug 2013 02:15:09 -0500
+Progress: time: Mon, 19 Aug 2013 02:15:22 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:15:26 -0500 Submitting:1 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 02:15:29 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:15:36 -0500 Submitted:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:15:37 -0500 Finished successfully:4
+real 37.97
+user 6.92
+sys 1.30
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+client: connect: Connection refused
+client: failed to connect
+client: connect: Connection refused
+client: failed to connect
+client: connect: Connection refused
+client: failed to connect
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0219-dk0ovle8
+Progress: time: Mon, 19 Aug 2013 02:19:30 -0500
+Progress: time: Mon, 19 Aug 2013 02:19:46 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:19:49 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:19:57 -0500 Submitted:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:19:58 -0500 Finished successfully:4
+real 37.61
+user 7.61
+sys 1.14
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+parcombiner_c_exec.sh: line 26: 28242 Segmentation fault (core dumped) ~/bin/exec_client $SITE recombiner.sh > $SITE.$ID.txt
+parcombiner_c_exec.sh: line 26: 28243 Segmentation fault (core dumped) ~/bin/exec_client $SITE recombiner.sh > $SITE.$ID.txt
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0228-lgkqd5s2
+Progress: time: Mon, 19 Aug 2013 02:28:40 -0500
+Progress: time: Mon, 19 Aug 2013 02:28:52 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:28:56 -0500 Submitting:1 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 02:29:00 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:29:07 -0500 Submitted:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:29:08 -0500 Finished successfully:4
+real 38.07
+user 6.25
+sys 1.13
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+parcombiner_c_exec.sh: line 26: 28242 Segmentation fault (core dumped) ~/bin/exec_client $SITE recombiner.sh > $SITE.$ID.txt
+parcombiner_c_exec.sh: line 26: 28243 Segmentation fault (core dumped) ~/bin/exec_client $SITE recombiner.sh > $SITE.$ID.txt
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0235-82pcyfnd
+Progress: time: Mon, 19 Aug 2013 02:35:33 -0500
+Progress: time: Mon, 19 Aug 2013 02:35:44 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:35:48 -0500 Submitting:1 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 02:35:52 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:35:59 -0500 Submitted:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:36:00 -0500 Finished successfully:4
+real 36.23
+user 6.11
+sys 1.09
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT : 30
+EXPECTED COUNT: 300000
+parcombiner_c_exec.sh: line 26: 28242 Segmentation fault (core dumped) ~/bin/exec_client $SITE recombiner.sh > $SITE.$ID.txt
+parcombiner_c_exec.sh: line 26: 28243 Segmentation fault (core dumped) ~/bin/exec_client $SITE recombiner.sh > $SITE.$ID.txt
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0247-fylobguf
+Progress: time: Mon, 19 Aug 2013 02:47:28 -0500
+Progress: time: Mon, 19 Aug 2013 02:47:40 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:47:41 -0500 Submitting:1 Active:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 02:47:44 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:47:51 -0500 Submitted:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:47:52 -0500 Finished successfully:4
+real 32.91
+user 6.14
+sys 0.93
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT : 20
+EXPECTED COUNT: 300000
+parcombiner_c_exec.sh: line 26: 28242 Segmentation fault (core dumped) ~/bin/exec_client $SITE recombiner.sh > $SITE.$ID.txt
+parcombiner_c_exec.sh: line 26: 28243 Segmentation fault (core dumped) ~/bin/exec_client $SITE recombiner.sh > $SITE.$ID.txt
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0248-k3i3jkug
+Progress: time: Mon, 19 Aug 2013 02:48:17 -0500
+Progress: time: Mon, 19 Aug 2013 02:48:30 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:48:34 -0500 Submitting:1 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 02:48:40 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:48:41 -0500 Stage out:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:48:41 -0500 Finished successfully:4
+real 34.26
+user 6.09
+sys 0.90
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0250-1ohnsl10
+Progress: time: Mon, 19 Aug 2013 02:50:11 -0500
+Progress: time: Mon, 19 Aug 2013 02:50:29 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:50:32 -0500 Submitting:1 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 02:50:37 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:50:41 -0500 Submitting:1 Finished successfully:3
+Progress: time: Mon, 19 Aug 2013 02:50:45 -0500 Submitted:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:50:45 -0500 Finished successfully:4
+real 45.03
+user 7.82
+sys 1.49
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0253-ncyt2tv9
+Progress: time: Mon, 19 Aug 2013 02:53:30 -0500 Initializing:1
+Progress: time: Mon, 19 Aug 2013 02:53:42 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:53:43 -0500 Submitting:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:53:45 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 02:53:46 -0500 Active:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:53:46 -0500 Finished successfully:4
+real 25.75
+user 5.80
+sys 0.73
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0258-gl5m2n62
+Progress: time: Mon, 19 Aug 2013 02:58:56 -0500
+Progress: time: Mon, 19 Aug 2013 02:59:09 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 02:59:11 -0500 Submitting:1 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 02:59:20 -0500 Submitted:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 02:59:20 -0500 Finished successfully:4
+real 35.29
+user 5.99
+sys 0.87
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0301-nqgfl574
+Progress: time: Mon, 19 Aug 2013 03:01:12 -0500
+Progress: time: Mon, 19 Aug 2013 03:01:24 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 03:01:27 -0500 Submitting:1 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 03:01:32 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 03:01:38 -0500 Submitted:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 03:01:39 -0500 Finished successfully:4
+real 37.01
+user 6.36
+sys 0.99
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+parcombiner_c_exec.sh: line 37: exec_client: command not found
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0302-m08fkkya
+Progress: time: Mon, 19 Aug 2013 03:02:24 -0500
+Progress: time: Mon, 19 Aug 2013 03:02:36 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 03:02:37 -0500 Submitting:1 Stage out:2
+Progress: time: Mon, 19 Aug 2013 03:02:39 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 03:02:41 -0500 Active:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 03:02:41 -0500 Finished successfully:4
+real 25.76
+user 5.67
+sys 0.66
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+cp: cannot create regular file `./exec_client': File exists
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:anl, CHUNKS:3 X FILESIZE:10
+Swift 0.94RC4 swift-r6303 cog-r3610
+
+RunID: 20130819-0304-5624h1v4
+Progress: time: Mon, 19 Aug 2013 03:04:28 -0500
+Progress: time: Mon, 19 Aug 2013 03:04:39 -0500 Submitting:2 Submitted:1
+Progress: time: Mon, 19 Aug 2013 03:04:43 -0500 Submitting:1 Submitted:1 Finished successfully:1
+Progress: time: Mon, 19 Aug 2013 03:04:46 -0500 Submitted:1 Finished successfully:2
+Progress: time: Mon, 19 Aug 2013 03:04:47 -0500 Active:1 Finished successfully:3
+Final status: Mon, 19 Aug 2013 03:04:48 -0500 Finished successfully:4
+real 29.97
+user 5.65
+sys 0.95
+chunks 3 filesize 10 site anl type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 300000
+cp: cp: cannot create regular file `./exec_client'cannot create regular file `./exec_client': Text file busy: Text file busy
+
+parcombiner_c_exec.sh: line 26: 24727 Segmentation fault (core dumped) ./exec_client $SITE $SITE.$ID.sh &>$SITE.$ID.txt
+---------------------------------------------
Modified: SwiftApps/Swift-MapRed/mapred_chirp_combiner/RESULT_FILE
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/RESULT_FILE 2013-08-16 23:22:26 UTC (rev 6878)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/RESULT_FILE 2013-08-19 15:37:33 UTC (rev 6879)
@@ -1 +1 @@
-thrash /sandbox/yadunand/input_10132.txt
+steamroller /sandbox/yadunand/input_30326.txt
Modified: SwiftApps/Swift-MapRed/mapred_chirp_combiner/TEMP
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/TEMP 2013-08-16 23:22:26 UTC (rev 6878)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/TEMP 2013-08-19 15:37:33 UTC (rev 6879)
@@ -1,9 +1 @@
-200309882511732 400000
-100123682586326 200000
-399951801425132 800000
-99882602648500 200000
-200214304630449 400000
-99939885782322 200000
-400334753729323 800000
-300142938339049 600000
-200290785918244 400000
+10 10
Added: SwiftApps/Swift-MapRed/mapred_chirp_combiner/client.c
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/client.c (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/client.c 2013-08-19 15:37:33 UTC (rev 6879)
@@ -0,0 +1,130 @@
+/*
+** client.c -- a stream socket client demo
+* Thanks to Brian "Beej Jorgensen" Hall
+* Hacked to work as a remote exec server by Yadu Nand B
+* <yadunandb at ci.uchicago.edu>
+*/
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <netdb.h>
+#include <sys/types.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+#include <string.h>
+
+#include <arpa/inet.h>
+
+#define PORT "3490" // the port client will be connecting to
+
+#define MAXDATASIZE 2048 // max number of bytes we can get at once
+
+// get sockaddr, IPv4 or IPv6:
+void *get_in_addr(struct sockaddr *sa)
+{
+ if (sa->sa_family == AF_INET) {
+ return &(((struct sockaddr_in*)sa)->sin_addr);
+ }
+
+ return &(((struct sockaddr_in6*)sa)->sin6_addr);
+}
+
+
+int main(int argc, char *argv[])
+{
+ int sockfd, new_fd, numbytes;
+ int filelen;
+ struct addrinfo hints, *servinfo, *p;
+ int rv;
+ char s[INET6_ADDRSTRLEN];
+ FILE *f_in;
+ int file_size;
+ char *buffer;
+ char *reply_buf;
+
+ if (argc != 3) {
+ fprintf(stderr,"usage: client hostname remote_script\n");
+ exit(1);
+ }
+
+ memset(&hints, 0, sizeof hints);
+ hints.ai_family = AF_UNSPEC;
+ hints.ai_socktype = SOCK_STREAM;
+
+ if ((rv = getaddrinfo(argv[1], PORT, &hints, &servinfo)) != 0) {
+ fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rv));
+ return 1;
+ }
+
+ // loop through all the results and connect to the first we can
+ for(p = servinfo; p != NULL; p = p->ai_next) {
+ if ((sockfd = socket(p->ai_family, p->ai_socktype,
+ p->ai_protocol)) == -1) {
+ perror("client: socket");
+ continue;
+ }
+
+ if (connect(sockfd, p->ai_addr, p->ai_addrlen) == -1) {
+ close(sockfd);
+ perror("client: connect");
+ continue;
+ }
+
+ break;
+ }
+
+ if (p == NULL) {
+ fprintf(stderr, "client: failed to connect\n");
+ return 2;
+ }
+
+ inet_ntop(p->ai_family, get_in_addr((struct sockaddr *)p->ai_addr),
+ s, sizeof s);
+ //printf("client: connecting to %s\n", s);
+
+ freeaddrinfo(servinfo); // all done with this structure
+
+ //Send message to server
+ //printf("Client sending contents of file %s to server \n",argv[1]);
+
+ // Read file contents into a buffer;
+ if ( (f_in = fopen(argv[2],"r")) == NULL ){
+ printf("Opening[%s] failed: %s\n", argv[2], strerror(errno));
+ exit(errno);
+ };
+
+ fseek(f_in, 0, SEEK_END);
+ file_size = ftell(f_in); // Find length
+ //printf("File size: %d\n", file_size);
+ fseek(f_in, 0, SEEK_SET);
+
+ buffer = (char *) malloc (file_size);
+ fread(buffer, file_size, 1, f_in);
+ buffer[file_size] = '\0';
+ //printf("String : %s\n", buffer);
+ fclose(f_in);
+
+ //printf("Content of file : \n%s\n", buffer);
+
+ if (send(sockfd, buffer, file_size, 0) == -1)
+ perror("send");
+ free(buffer);
+
+ //Receive message back from server
+ reply_buf = (char *)malloc(MAXDATASIZE);
+ //printf("Client receiving message from server \n");
+ if ((numbytes = recv(sockfd, reply_buf, MAXDATASIZE-1, 0)) == -1) {
+ perror("recv");
+ exit(1);
+ }
+
+ reply_buf[numbytes] = '\0';
+ printf("%s",reply_buf);
+ free(reply_buf);
+ close(sockfd);
+
+ return 0;
+}
Added: SwiftApps/Swift-MapRed/mapred_chirp_combiner/compile
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/compile (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/compile 2013-08-19 15:37:33 UTC (rev 6879)
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+gcc server.c -o exec_server
+gcc client.c -o exec_client
+
+chmod a+x exec_server
+chmod a+x exec_client
Property changes on: SwiftApps/Swift-MapRed/mapred_chirp_combiner/compile
___________________________________________________________________
Added: svn:executable
+ *
Modified: SwiftApps/Swift-MapRed/mapred_chirp_combiner/err_file
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/err_file 2013-08-16 23:22:26 UTC (rev 6878)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/err_file 2013-08-19 15:37:33 UTC (rev 6879)
@@ -1,4 +1,3 @@
-chirp_put: couldn't put ./recombiner.sh to host crank: Permission denied
-couldn't get crank:crank.17255.imd: Permission denied
-cat: crank.17255.txt: No such file or directory
-cat: crank.17255.txt: No such file or directory
+cp: cp: cannot create regular file `./exec_client'cannot create regular file `./exec_client': Text file busy: Text file busy
+
+parcombiner_c_exec.sh: line 26: 24727 Segmentation fault (core dumped) ./exec_client $SITE $SITE.$ID.sh &>$SITE.$ID.txt
Modified: SwiftApps/Swift-MapRed/mapred_chirp_combiner/final_result
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/final_result 2013-08-16 23:22:26 UTC (rev 6878)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/final_result 2013-08-19 15:37:33 UTC (rev 6879)
@@ -1,2 +1,5 @@
-Final results (on churn):
-5000738965127778 10000000
+Content of stomp.20006.txt is
+Content of vanquish.11894.txt is
+Content of thrash.9680.txt is
+Final results (on stomp):
+
Added: SwiftApps/Swift-MapRed/mapred_chirp_combiner/parcombiner_c_exec.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/parcombiner_c_exec.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/parcombiner_c_exec.sh 2013-08-19 15:37:33 UTC (rev 6879)
@@ -0,0 +1,58 @@
+#!/bin/bash
+
+FILES=$*
+SUM=0
+COUNT=0
+REMOTE_PORT=29900
+cat $* > RESULT_FILE
+
+# Find the unique sites in the RESULT_FILE to run a combiner there.
+#SITES=($(sort -u -t' ' -k1,1 RESULT_FILE))
+cat <<'EOF' > recombiner.sh
+SUM=0;COUNT=0;
+for file in ${FILES[*]}; do
+ [ ! -f $file ] && echo "$file is missing" >> $HOSTNAME.parrot_run.log
+ RES=($(awk '{ sum += $1 } END { print sum,NR }' $file))
+ SUM=$(($SUM+${RES[0]}))
+ COUNT=$(($COUNT+${RES[1]}))
+ rm $file &
+done;
+echo -e "$SUM $COUNT"
+EOF
+chmod 777 recombiner.sh
+
+
+combine_fetch_process()
+{
+ SITE=$1; shift
+ ID=$1; shift
+ REMOTE_PORT=$1;
+ PORT=$(($REMOTE_PORT+$ID))
+
+ FILES=($(grep $SITE RESULT_FILE | awk '{ print $2 }'))
+ CHIRPDIR=$(dirname ${FILES[0]})
+
+ echo -e "#!/bin/bash \nID=$ID \nFILES=(${FILES[*]}) \n" > $SITE.$ID.sh
+ cat recombiner.sh >> $SITE.$ID.sh
+ cp /homes/yadunand/bin/exec_client ./
+ ./exec_client $SITE $SITE.$ID.sh &> $SITE.$ID.txt
+ echo "Content of $SITE.$ID.txt is $(cat $SITE.$ID.txt)"
+}
+
+STAGE2="";
+SITES=($(awk '{ print $1 }' RESULT_FILE | sort -u))
+for SITE in ${SITES[*]}
+do
+ ID=$RANDOM
+ combine_fetch_process $SITE $ID $REMOTE_PORT &
+ STAGE2="$STAGE2 $SITE.$ID.txt"
+done;
+wait
+
+cat ${STAGE2[*]} > TEMP
+SUM=($(awk '{ sum += $1 } END { print sum }' TEMP))
+COUNT=($(awk '{ sum += $2 } END { print sum }' TEMP))
+echo "Final results (on $HOSTNAME):"
+echo "$SUM $COUNT"
+
+exit 0;
\ No newline at end of file
Property changes on: SwiftApps/Swift-MapRed/mapred_chirp_combiner/parcombiner_c_exec.sh
___________________________________________________________________
Added: svn:executable
+ *
Modified: SwiftApps/Swift-MapRed/mapred_chirp_combiner/recombiner.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/recombiner.sh 2013-08-16 23:22:26 UTC (rev 6878)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/recombiner.sh 2013-08-19 15:37:33 UTC (rev 6879)
@@ -4,5 +4,6 @@
RES=($(awk '{ sum += $1 } END { print sum,NR }' $file))
SUM=$(($SUM+${RES[0]}))
COUNT=$(($COUNT+${RES[1]}))
+ rm $file &
done;
echo -e "$SUM $COUNT"
Modified: SwiftApps/Swift-MapRed/mapred_chirp_combiner/run.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/run.sh 2013-08-16 23:22:26 UTC (rev 6878)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/run.sh 2013-08-19 15:37:33 UTC (rev 6879)
@@ -13,6 +13,12 @@
SITE="anl"
fi
+if [ "$SITE" == "beagle" ]
+then
+ ./compile
+ cp exec_server /lustre/beagle/yadunandb/bin/
+ cp exec_client /lustre/beagle/yadunandb/bin/
+fi;
#Call as check_error $? <ERR_CODE> <Error message>
check_error()
{
@@ -74,7 +80,7 @@
echo "================================================================"
}
-run_swift_chirp_combiner 20 50;
+run_swift_chirp_combiner 3 10;
exit 0
####################SWIFT + CHIRP VERSION############################
Added: SwiftApps/Swift-MapRed/mapred_chirp_combiner/server.c
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/server.c (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/server.c 2013-08-19 15:37:33 UTC (rev 6879)
@@ -0,0 +1,183 @@
+/*
+** server.c -- a stream socket server demo
+*/
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <netdb.h>
+#include <arpa/inet.h>
+#include <sys/wait.h>
+#include <signal.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#define PORT "3490" // the port users will be connecting to
+#define MAXDATASIZE 2048
+#define BACKLOG 10 // how many pending connections queue will hold
+
+void sigchld_handler(int s)
+{
+ while(waitpid(-1, NULL, WNOHANG) > 0);
+}
+
+// get sockaddr, IPv4 or IPv6:
+void *get_in_addr(struct sockaddr *sa)
+{
+ if (sa->sa_family == AF_INET) {
+ return &(((struct sockaddr_in*)sa)->sin_addr);
+ }
+
+ return &(((struct sockaddr_in6*)sa)->sin6_addr);
+}
+
+
+int exec_string(char *query, int query_size, char *reply)
+{
+ FILE *p_exec;
+ FILE *f_out;
+ char filename[100];
+ char exec_string[500];
+ int ret;
+
+ sprintf(filename,"exec_script_%d_%d", getpid(), rand());
+ //printf("Debug: filename: %s \n", filename);
+
+ if ( (f_out = fopen(filename,"w")) == NULL ){
+ printf("Opening[%s] failed: %s\n", filename, strerror(errno));
+ exit(errno);
+ };
+ fwrite(query, query_size, 1, f_out);
+ fclose(f_out);
+
+ sprintf(exec_string, "chmod +x %s; ./%s",
+ filename, filename);
+ //printf("Command string : [%s]\n", exec_string);
+
+ if ( (p_exec = popen(exec_string, "r")) == NULL ){
+ printf("exec_string failed to popen: %s", strerror(errno));
+ return errno;
+ }
+
+ ret = fread(reply, MAXDATASIZE, 1, p_exec);
+
+ //printf("Reply is: \n%s Len = %d\n", reply, (int)strlen(reply));
+ return (strlen(reply));
+}
+
+int main(void)
+{
+ int sockfd, new_fd; // listen on sock_fd, new connection on new_fd
+ struct addrinfo hints, *servinfo, *p;
+ struct sockaddr_storage their_addr; // connector's address information
+ socklen_t sin_size;
+ struct sigaction sa;
+ int yes=1;
+ char s[INET6_ADDRSTRLEN];
+ int rv;
+ char buf[MAXDATASIZE];
+ int numbytes;
+ char *repy;
+
+ memset(&hints, 0, sizeof hints);
+ hints.ai_family = AF_UNSPEC;
+ hints.ai_socktype = SOCK_STREAM;
+ hints.ai_flags = AI_PASSIVE; // use my IP
+
+ if ((rv = getaddrinfo(NULL, PORT, &hints, &servinfo)) != 0) {
+ fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rv));
+ return 1;
+ }
+
+ // loop through all the results and bind to the first we can
+ for(p = servinfo; p != NULL; p = p->ai_next) {
+ if ((sockfd = socket(p->ai_family, p->ai_socktype,
+ p->ai_protocol)) == -1) {
+ perror("server: socket");
+ continue;
+ }
+
+ if (setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &yes,
+ sizeof(int)) == -1) {
+ perror("setsockopt");
+ exit(1);
+ }
+
+ if (bind(sockfd, p->ai_addr, p->ai_addrlen) == -1) {
+ close(sockfd);
+ perror("server: bind");
+ continue;
+ }
+
+ break;
+ }
+
+ if (p == NULL) {
+ fprintf(stderr, "server: failed to bind\n");
+ return 2;
+ }
+
+ freeaddrinfo(servinfo); // all done with this structure
+
+ if (listen(sockfd, BACKLOG) == -1) {
+ perror("listen");
+ exit(1);
+ }
+
+ sa.sa_handler = sigchld_handler; // reap all dead processes
+ sigemptyset(&sa.sa_mask);
+ sa.sa_flags = SA_RESTART;
+ if (sigaction(SIGCHLD, &sa, NULL) == -1) {
+ perror("sigaction");
+ exit(1);
+ }
+
+ //printf("server: waiting for connections...\n");
+
+ while(1) { // main accept() loop
+ sin_size = sizeof their_addr;
+ new_fd = accept(sockfd, (struct sockaddr *)&their_addr, &sin_size);
+ if (new_fd == -1) {
+ perror("accept");
+ continue;
+ }
+
+ inet_ntop(their_addr.ss_family,
+ get_in_addr((struct sockaddr *)&their_addr),
+ s, sizeof s);
+ printf("server: got connection from %s\n", s);
+
+ if (!fork()) { // this is the child process
+ int reply_size;
+ char *reply;
+ close(sockfd); // child doesn't need the listener
+ if ((numbytes = recv(new_fd, buf, MAXDATASIZE-1, 0)) == -1) {
+ perror("recv");
+ exit(1);
+ }
+
+ buf[numbytes] = '\0';
+ //printf("Server: Received string from client: \n'%s'\n",buf);
+
+ // TODO
+ reply = (char *) malloc (2048);
+ reply_size = exec_string(buf, numbytes, reply);
+ //printf("Returning string of size %d \n%s\n", reply_size, reply);
+
+ if (send(new_fd, reply, reply_size, 0) == -1)
+ perror("send");
+ //printf("Server: Send string 'Hello, client'\n");
+ close(new_fd);
+ free(reply);
+ exit(0);
+ }
+ close(new_fd); // parent doesn't need this
+ return 0;
+ }
+ return 0;
+}
Modified: SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen.kml
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen.kml 2013-08-16 23:22:26 UTC (rev 6878)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen.kml 2013-08-19 15:37:33 UTC (rev 6879)
@@ -149,35 +149,35 @@
</set>
<set name="wrapper">
- <vdl:new type="script" dbgname="wrapper" _defline="18" input="true">
+ <vdl:new type="script" dbgname="wrapper" _defline="19" input="true">
<vdl:mapping descriptor="single_file_mapper">
- <vdl:parameter name="file" value="teragen_wrap_nc.sh"/>
+ <vdl:parameter name="file" value="teragen_wrap_c_exec.sh"/>
</vdl:mapping>
</vdl:new>
</set>
<set name="loop">
- <vdl:new type="int" dbgname="loop" waitCount="1" _defline="19"/>
+ <vdl:new type="int" dbgname="loop" waitCount="1" _defline="20"/>
</set>
<set name="fsize">
- <vdl:new type="int" dbgname="fsize" waitCount="1" _defline="20"/>
+ <vdl:new type="int" dbgname="fsize" waitCount="1" _defline="21"/>
</set>
<set name="dir">
- <vdl:new type="string" dbgname="dir" waitCount="1" _defline="21"/>
+ <vdl:new type="string" dbgname="dir" waitCount="1" _defline="22"/>
</set>
<set name="combine">
- <vdl:new type="script" dbgname="combine" _defline="28" input="true">
+ <vdl:new type="script" dbgname="combine" _defline="30" input="true">
<vdl:mapping descriptor="single_file_mapper">
- <vdl:parameter name="file" value="parcombiner_nc.sh"/>
+ <vdl:parameter name="file" value="parcombiner_c_exec.sh"/>
</vdl:mapping>
</vdl:new>
</set>
<set name="final">
- <vdl:new type="file" dbgname="final" _defline="29">
+ <vdl:new type="file" dbgname="final" _defline="31">
<vdl:mapping descriptor="single_file_mapper">
<vdl:parameter name="file" value="final_result"/>
</vdl:mapping>
@@ -185,7 +185,7 @@
</set>
<set name="errs">
- <vdl:new type="file" dbgname="errs" _defline="30">
+ <vdl:new type="file" dbgname="errs" _defline="32">
<vdl:mapping descriptor="single_file_mapper">
<vdl:parameter name="file" value="err_file"/>
</vdl:mapping>
@@ -226,10 +226,10 @@
</sequential>
<sequential>
- <vdl:setfieldvalue _traceline="20">
+ <vdl:setfieldvalue _traceline="21">
<variable>loop</variable>
- <swiftscript:to_int _traceline="19">
- <swiftscript:arg _traceline="19">
+ <swiftscript:to_int _traceline="20">
+ <swiftscript:arg _traceline="20">
<variable>swift#string#17007</variable><variable>swift#string#17008</variable>
</swiftscript:arg>
</swiftscript:to_int>
@@ -237,10 +237,10 @@
<partialCloseDataset var="{loop}"/>
</sequential>
<sequential>
- <vdl:setfieldvalue _traceline="21">
+ <vdl:setfieldvalue _traceline="22">
<variable>fsize</variable>
- <swiftscript:to_int _traceline="20">
- <swiftscript:arg _traceline="20">
+ <swiftscript:to_int _traceline="21">
+ <swiftscript:arg _traceline="21">
<variable>swift#string#17009</variable><variable>swift#string#17010</variable>
</swiftscript:arg>
</swiftscript:to_int>
@@ -248,9 +248,9 @@
<partialCloseDataset var="{fsize}"/>
</sequential>
<sequential>
- <vdl:setfieldvalue _traceline="23">
+ <vdl:setfieldvalue _traceline="24">
<variable>dir</variable>
- <swiftscript:arg _traceline="21">
+ <swiftscript:arg _traceline="22">
<variable>swift#string#17011</variable><variable>swift#string#17012</variable>
</swiftscript:arg>
</vdl:setfieldvalue>
@@ -279,10 +279,10 @@
<set name="i">
<vdl:new type="int" value="{$$}"/>
</set>
- <unitStart line="23" type="FOREACH_IT"/>
+ <unitStart line="24" type="FOREACH_IT"/>
<sequentialWithID>
<sequential>
- <gen__data _traceline="24">
+ <gen__data _traceline="25">
<vdl:getfieldsubscript>
<argument name="var"><variable>tgen_out</variable></argument>
<argument name="subscript"><variable>i</variable></argument>
@@ -297,11 +297,11 @@
<partialCloseDataset var="{tgen_out}"/><partialCloseDataset var="{tgen_err}"/>
</sequential>
</sequentialWithID>
- <unitEnd line="23" type="FOREACH_IT"/>
+ <unitEnd line="24" type="FOREACH_IT"/>
</vdl:tparallelFor>
</sequential>
<sequential>
- <comb__data _traceline="31">
+ <comb__data _traceline="33">
<variable>final</variable>
<variable>errs</variable>
<variable>combine</variable>
Modified: SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen.swift
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen.swift 2013-08-16 23:22:26 UTC (rev 6878)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen.swift 2013-08-19 15:37:33 UTC (rev 6879)
@@ -15,7 +15,8 @@
file tgen_out[] <simple_mapper; prefix="tgen", suffix=".out">;
file tgen_err[] <simple_mapper; prefix="tgen", suffix=".err">;
-script wrapper <"teragen_wrap_nc.sh">;
+//script wrapper <"teragen_wrap_nc.sh">;
+script wrapper <"teragen_wrap_c_exec.sh">;
int loop = @toInt(@arg("loops","0"));
int fsize = @toInt(@arg("recsize","10")); # This would make 10M records per file
string dir = @arg("dir", "./");
@@ -25,7 +26,8 @@
}
-script combine <"parcombiner_nc.sh">;
+//script combine <"parcombiner_nc.sh">;
+script combine <"parcombiner_c_exec.sh">;
file final <"final_result">;
file errs <"err_file">;
(final, errs) = comb_data(combine, tgen_out);
Added: SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen_wrap_c_exec.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen_wrap_c_exec.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen_wrap_c_exec.sh 2013-08-19 15:37:33 UTC (rev 6879)
@@ -0,0 +1,46 @@
+#!/bin/bash
+
+
+# By default with ARG1:100 and SLICESIZE=10000, this script will generate
+# 10^6 records.
+ARG1=1
+[ ! -z $1 ] && ARG1=$1
+
+FILE="input_$RANDOM.txt"
+LOWERLIMIT=0
+UPPERLIMIT=1000000000 # 10^9
+SLICESIZE=10000 # 10^4 records padded to 100B would result in 1MB file
+#SLICESIZE=1000 # 10^3 If padded to 100B would result
+
+
+FOLDER="/sandbox/$USER"
+TIMEOUT=1200
+NC_LISTEN_PORT=29900
+
+# clean_folder SLEEP_TIME FOLDER_NAME
+clean_folder()
+{
+ sleep $(($1+1));
+ rm $2/* &> /dev/null
+}
+
+JOBS_LEVEL=`echo $PWD | grep -o ".*jobs"`
+if [ "$?" == "0" ]
+then
+ mkdir $JOBS_LEVEL/CHIRPING
+ if [ "$?" == "0" ]
+ then
+ killall -u $USER chirp_server;
+ echo "unix:$USER rwlds" > $FOLDER/acl.conf
+ echo "hostname:* rwl" >> $FOLDER/acl.conf
+ timeout $TIMEOUT chirp_server -A $FOLDER/acl.conf -r $FOLDER &
+ timeout $TIMEOUT ~/bin/exec_server &
+ fi
+ FILE=$FOLDER/$FILE;
+else
+ FILE=$PWD/$FILE
+fi
+
+shuf -i $LOWERLIMIT-$UPPERLIMIT -n $(($SLICESIZE*$ARG1)) | awk '{printf "%-99s\n", $0}' > $FILE
+echo "$HOSTNAME $FILE"
+exit 0
\ No newline at end of file
Property changes on: SwiftApps/Swift-MapRed/mapred_chirp_combiner/teragen_wrap_c_exec.sh
___________________________________________________________________
Added: svn:executable
+ *
More information about the Swift-commit
mailing list