[Swift-commit] r7148 - in SwiftApps/Swift-MapRed: . mapred_combiner_V4
yadunandb at ci.uchicago.edu
yadunandb at ci.uchicago.edu
Thu Oct 10 22:25:32 CDT 2013
Author: yadunandb
Date: 2013-10-10 22:25:32 -0500 (Thu, 10 Oct 2013)
New Revision: 7148
Added:
SwiftApps/Swift-MapRed/mapred_combiner_V4/
SwiftApps/Swift-MapRed/mapred_combiner_V4/LAB_RECORDS
SwiftApps/Swift-MapRed/mapred_combiner_V4/README
SwiftApps/Swift-MapRed/mapred_combiner_V4/aggregate.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/beagle.xml
SwiftApps/Swift-MapRed/mapred_combiner_V4/clean.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/client.c
SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner_template.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/compile
SwiftApps/Swift-MapRed/mapred_combiner_V4/dereference.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/map_core.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper_template.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/midway.xml
SwiftApps/Swift-MapRed/mapred_combiner_V4/multiple.xml
SwiftApps/Swift-MapRed/mapred_combiner_V4/node_uniq.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/reduce_core.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/reducer_template.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/run.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/server.c
SwiftApps/Swift-MapRed/mapred_combiner_V4/setup.sh
SwiftApps/Swift-MapRed/mapred_combiner_V4/swift.properties
SwiftApps/Swift-MapRed/mapred_combiner_V4/swiftreduce.swift
SwiftApps/Swift-MapRed/mapred_combiner_V4/tc.data
Log:
Committing next version of SwiftReduce.
Functionality moved out of swift to app scripts to get better
performance.
Only simple reduce mechanisms implemented.
Tree reduction and local combiner operations are pending.
Works across multiple sites. Tested across beagle and midway.
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/LAB_RECORDS
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/LAB_RECORDS (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/LAB_RECORDS 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,3207 @@
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:2 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Could not start execution
+ Compile error in foreach statement at line 93
+ Compile error in procedure invocation at line 94
+ Variable comb_out was not declared in this scope.
+real 1.92
+user 2.84
+sys 0.09
+chunks 2 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 200000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Could not start execution
+ Compile error in foreach statement at line 93
+ Compile error in procedure invocation at line 94
+ Variable comb_out was not declared in this scope.
+real 2.56
+user 2.81
+sys 0.11
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131007-1742-up4o3it5
+Progress: time: Mon, 07 Oct 2013 17:42:27 -0500
+Multiple entries found for bash on midway. Using the first one
+Execution failed:
+ Exception in bash:
+ Arguments: [mapper.sh, 10]
+ Host: midway
+ Directory: teragen-20131007-1742-up4o3it5/jobs/l/bash-lmrzbcgl
+
+Caused by:
+ Could not submit job
+Caused by:
+ Could not start coaster service
+Caused by:
+ No 'ssh-c' provider or alias found. Available providers: [cobalt, ssh-cl, gsiftp, coaster, webdav, slurm, dcache, ssh, gt2, condor, http, coaster-persistent, pbs, ftp, lsf, gsiftp-old, local, sge]. Aliases: condor <-> condorlocal; cobalt <-> cobaltlocal; gsiftp <-> gridftp; pbs <-> pbslocal; gsiftp-old <-> gridftp-old; local <-> file;
+ map_data, teragen.swift, line 84
+real 4.22
+user 7.48
+sys 0.26
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131007-1742-a3ivkkqb
+Progress: time: Mon, 07 Oct 2013 17:42:50 -0500
+Multiple entries found for bash on midway. Using the first one
+Progress: time: Mon, 07 Oct 2013 17:42:56 -0500 Submitting:19 Submitted:1
+Execution failed:
+ Exception in bash:
+ Arguments: [mapper.sh, 10]
+ Host: midway
+ Directory: teragen-20131007-1742-a3ivkkqb/jobs/v/bash-v5c0ccgl
+
+Caused by:
+ Could not submit job
+Caused by:
+ Could not start coaster service
+Caused by:
+ Task ended before registration was received.
+Caused by:
+ org.globus.cog.abstraction.impl.common.task.InvalidSecurityContextException: org.globus.gsi.GlobusCredentialException: [JGLOBUS-5] Proxy file (/lustre/beagle/yadunandb/.globus/coasters/proxy.0.pem) not found.
+ map_data, teragen.swift, line 84
+real 10.27
+user 16.00
+sys 4.06
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:beagle, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131008-1652-fcxkru50
+Progress: time: Tue, 08 Oct 2013 16:52:08 -0500
+Execution failed:
+ The application "bash" is not available for any site/pool in your tc.data catalog
+ map_data, teragen.swift, line 84
+real 4.09
+user 7.04
+sys 0.14
+chunks 20 filesize 10 site beagle type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131008-1652-muvj1kfe
+Progress: time: Tue, 08 Oct 2013 16:52:40 -0500
+Execution failed:
+ Exception in bash:
+ Arguments: [mapper.sh, 10]
+ Host: midway
+ Directory: teragen-20131008-1652-muvj1kfe/jobs/t/bash-tmakxdgl
+
+Caused by:
+ Could not submit job
+Caused by:
+ Could not start coaster service
+Caused by:
+ java.lang.NullPointerException
+ at org.globus.cog.abstraction.impl.execution.coaster.AutoCA.ensureCACertsExist(AutoCA.java:197)
+ at org.globus.cog.abstraction.impl.execution.coaster.AutoCA.createProxy(AutoCA.java:128)
+ at org.globus.cog.abstraction.impl.execution.coaster.ServiceManager.setupGSIProxy(ServiceManager.java:238)
+ at org.globus.cog.abstraction.impl.execution.coaster.ServiceManager.startService(ServiceManager.java:194)
+ at org.globus.cog.abstraction.impl.execution.coaster.ServiceManager.reserveService(ServiceManager.java:132)
+ at org.globus.cog.abstraction.impl.execution.coaster.ServiceManager.reserveService(ServiceManager.java:151)
+ at org.globus.cog.abstraction.impl.execution.coaster.JobSubmissionTaskHandler.getChannel(JobSubmissionTaskHandler.java:119)
+ at org.globus.cog.abstraction.impl.execution.coaster.JobSubmissionTaskHandler.submit(JobSubmissionTaskHandler.java:105)
+ at org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
+ at org.globus.cog.karajan.scheduler.submitQueue.NonBlockingSubmit.run(NonBlockingSubmit.java:97)
+ at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
+ at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
+ at java.util.concurrent.FutureTask.run(FutureTask.java:166)
+ at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
+ at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
+ at java.lang.Thread.run(Thread.java:722)
+
+ map_data, teragen.swift, line 84
+real 5.20
+user 9.90
+sys 0.47
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131008-2201-2i709u42
+Progress: time: Tue, 08 Oct 2013 22:01:13 +0000
+Progress: time: Tue, 08 Oct 2013 22:01:14 +0000 Stage in:15 Submitted:4 Active:1
+Node info for 0 : midway418.rcc.local /dev/shm/input_29176.txt /dev/shm/input_29664.txt /dev/shm/input_23159.txt /dev/shm/input_19319.txt /dev/shm/input_12204.txt /dev/shm/input_13646.txt /dev/shm/input_15594.txt /dev/shm/input_6501.txt /dev/shm/input_32749.txt /dev/shm/input_10230.txt /dev/shm/input_28751.txt /dev/shm/input_24530.txt /dev/shm/input_5112.txt /dev/shm/input_5154.txt /dev/shm/input_24906.txt /dev/shm/input_18108.txt /dev/shm/input_11908.txt /dev/shm/input_21176.txt /dev/shm/input_10768.txt /dev/shm/input_20542.txtFinal status: Tue, 08 Oct 2013 22:01:15 +0000 Finished successfully:21
+real 5.65
+user 8.03
+sys 0.79
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple.xml, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Could not open: multiple.xml.xml
+Execution failed:
+
+ Could not find sites file: multiple.xml.xml
+
+real 1.77
+user 3.13
+sys 0.14
+chunks 20 filesize 10 site multiple.xml type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131008-2203-56tckuf4
+Progress: time: Tue, 08 Oct 2013 22:03:41 +0000
+Progress: time: Tue, 08 Oct 2013 22:03:42 +0000 Stage in:1 Submitted:2 Stage out:10 Finished successfully:7
+Node info for 0 : midway417.rcc.local /dev/shm/input_19507.txt /dev/shm/input_9859.txt /dev/shm/input_19742.txt /dev/shm/input_11204.txt /dev/shm/input_16270.txt /dev/shm/input_12384.txt /dev/shm/input_31043.txt /dev/shm/input_14123.txt /dev/shm/input_3398.txt /dev/shm/input_2284.txt /dev/shm/input_729.txt /dev/shm/input_30943.txt /dev/shm/input_2754.txt /dev/shm/input_26225.txt /dev/shm/input_2583.txt /dev/shm/input_11968.txt /dev/shm/input_2725.txt /dev/shm/input_8242.txt /dev/shm/input_10152.txt /dev/shm/input_15825.txtFinal status: Tue, 08 Oct 2013 22:03:43 +0000 Finished successfully:21
+real 4.02
+user 7.70
+sys 0.59
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131008-2220-05v83c05
+Progress: time: Tue, 08 Oct 2013 22:20:24 +0000
+Progress: time: Tue, 08 Oct 2013 22:20:54 +0000 Submitted:10
+Progress: time: Tue, 08 Oct 2013 22:20:55 +0000 Submitted:8 Stage out:1 Finished successfully:1
+Progress: time: Tue, 08 Oct 2013 22:20:56 +0000 Stage out:1 Finished successfully:10
+Node info for 0 : midway417.rcc.local /dev/shm/input_10212.txt /dev/shm/input_23513.txt /dev/shm/input_2555.txt /dev/shm/input_9641.txt /dev/shm/input_25651.txt /dev/shm/input_12694.txt /dev/shm/input_18506.txt /dev/shm/input_18837.txt /dev/shm/input_13695.txt /dev/shm/input_5517.txtFinal status: Tue, 08 Oct 2013 22:20:57 +0000 Finished successfully:11
+real 34.91
+user 62.12
+sys 12.85
+chunks 10 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 1000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:10
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131008-2221-0m0czkka
+Progress: time: Tue, 08 Oct 2013 22:21:36 +0000
+Progress: time: Tue, 08 Oct 2013 22:21:37 +0000 Submitted:4 Stage out:1 Finished successfully:5
+Node info for 0 : midway009.rcc.local /dev/shm/input_18212.txt /dev/shm/input_21154.txt /dev/shm/input_152.txt /dev/shm/input_4304.txt /dev/shm/input_3581.txt /dev/shm/input_9333.txt /dev/shm/input_31034.txt /dev/shm/input_25106.txt /dev/shm/input_6121.txt /dev/shm/input_5673.txtFinal status: Tue, 08 Oct 2013 22:21:38 +0000 Finished successfully:11
+real 4.08
+user 7.34
+sys 0.73
+chunks 10 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 1000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:5
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1553-33m404yf
+Progress: time: Wed, 09 Oct 2013 15:53:31 +0000
+Progress: time: Wed, 09 Oct 2013 15:53:32 +0000 Submitted:7 Stage out:1 Finished successfully:2
+Node info for 0 : midway009.rcc.local /dev/shm/input_27105.txt /dev/shm/input_32062.txt /dev/shm/input_6217.txt /dev/shm/input_1760.txt /dev/shm/input_7129.txt /dev/shm/input_10871.txt /dev/shm/input_19255.txt /dev/shm/input_12548.txt /dev/shm/input_4846.txt /dev/shm/input_5610.txtFinal status: Wed, 09 Oct 2013 15:53:32 +0000 Finished successfully:11
+real 5.13
+user 7.74
+sys 0.74
+chunks 10 filesize 5 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 500000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:3 X FILESIZE:5
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1602-u3j8ftx6
+Progress: time: Wed, 09 Oct 2013 16:02:30 +0000
+Node info for 0 : midway004.rcc.local /dev/shm/input_1292.txt /dev/shm/input_19364.txt /dev/shm/input_8364.txtFinal status: Wed, 09 Oct 2013 16:02:31 +0000 Finished successfully:4
+real 4.27
+user 6.01
+sys 0.48
+chunks 3 filesize 5 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 150000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:2 X FILESIZE:2
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1604-aq3uaadf
+Progress: time: Wed, 09 Oct 2013 16:04:32 +0000
+Node info for 0 : midway004.rcc.local /dev/shm/input_31748.txt /dev/shm/input_11173.txtFinal status: Wed, 09 Oct 2013 16:04:33 +0000 Finished successfully:3
+real 3.42
+user 5.73
+sys 0.38
+chunks 2 filesize 2 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 40000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:2 X FILESIZE:2
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1615-ccqiidr7
+Progress: time: Wed, 09 Oct 2013 16:15:22 +0000
+Node info for 0 : midway009.rcc.local /dev/shm/input_17354.txt /dev/shm/input_18100.txtFinal status: Wed, 09 Oct 2013 16:15:23 +0000 Finished successfully:3
+real 4.70
+user 6.24
+sys 0.54
+chunks 2 filesize 2 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 40000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:2 X FILESIZE:2
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1625-e26lg25c
+Progress: time: Wed, 09 Oct 2013 16:25:48 +0000
+Node info for 0 : midway_midway009.rcc.local /dev/shm/input_31360.txt /dev/shm/input_9042.txtFinal status: Wed, 09 Oct 2013 16:25:49 +0000 Finished successfully:3
+real 4.29
+user 5.86
+sys 0.45
+chunks 2 filesize 2 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 40000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:2 X FILESIZE:2
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1645-97pocioc
+Progress: time: Wed, 09 Oct 2013 16:45:41 +0000
+Progress: time: Wed, 09 Oct 2013 16:45:42 +0000 Stage out:1 Finished successfully:1
+Node info for 0 : midway midway004.rcc.local /dev/shm/input_26589.txt /dev/shm/input_17236.txtFinal status: Wed, 09 Oct 2013 16:45:42 +0000 Finished successfully:3
+real 4.43
+user 6.49
+sys 0.57
+chunks 2 filesize 2 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 40000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:3 X FILESIZE:1
+Could not compile SwiftScript source: line 99:5: expecting '}', found 'else'
+real 0.43
+user 0.58
+sys 0.04
+chunks 3 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 30000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:3 X FILESIZE:1
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1705-de2ef5i0
+Progress: time: Wed, 09 Oct 2013 17:05:35 +0000
+Progress: time: Wed, 09 Oct 2013 17:05:36 +0000 Submitted:1 Stage out:1 Finished successfully:1
+Node info for 0 : midway midway011.rcc.local /dev/shm/input_22712.txt /dev/shm/input_28170.txt /dev/shm/input_17471.txtOn midway
+Final status: Wed, 09 Oct 2013 17:05:36 +0000 Finished successfully:4
+real 3.93
+user 6.71
+sys 0.59
+chunks 3 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 30000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1709-mcklm3x4
+Progress: time: Wed, 09 Oct 2013 17:09:54 +0000
+Progress: time: Wed, 09 Oct 2013 17:10:24 +0000 Submitting:10
+Progress: time: Wed, 09 Oct 2013 17:10:54 +0000 Submitting:10
+Progress: time: Wed, 09 Oct 2013 17:11:24 +0000 Submitting:10
+Progress: time: Wed, 09 Oct 2013 17:11:54 +0000 Submitting:10
+Progress: time: Wed, 09 Oct 2013 17:12:24 +0000 Submitting:10
+Progress: time: Wed, 09 Oct 2013 17:12:54 +0000 Submitting:10
+Progress: time: Wed, 09 Oct 2013 17:13:02 +0000 Submitting:9 Submitted:1
+Progress: time: Wed, 09 Oct 2013 17:13:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:13:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:14:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:14:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:15:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:15:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:16:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:16:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:17:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:17:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:18:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:18:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:19:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:19:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:20:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:20:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:21:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:21:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:22:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:22:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:23:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:23:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:24:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:24:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:25:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:25:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:26:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:26:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:27:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:27:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:28:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:28:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:29:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:29:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:30:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:30:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:31:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:31:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:32:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:32:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:33:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:33:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:34:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:34:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:35:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:35:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:36:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:36:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:37:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:37:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:38:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:38:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:39:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:39:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:40:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:40:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:41:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:41:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:42:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:42:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:43:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:43:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:44:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:44:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:45:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:45:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:46:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:46:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:47:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:47:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:48:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:48:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:49:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:49:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:50:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:50:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:51:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:51:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:52:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:52:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:53:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:53:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:54:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:54:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:55:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:55:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:56:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:56:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:57:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:57:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:58:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:58:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:59:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 17:59:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:00:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:00:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:01:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:01:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:02:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:02:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:03:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:03:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:04:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:04:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:05:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:05:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:06:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:06:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:07:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:07:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:08:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:08:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:09:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:09:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:10:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:10:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:11:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:11:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:12:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:12:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:13:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:13:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:14:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:14:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:15:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:15:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:16:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:16:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:17:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:17:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:18:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:18:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:19:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:19:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:20:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:20:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:21:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:21:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:22:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:22:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:23:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:23:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:24:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:24:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:25:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:25:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:26:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:26:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:27:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:27:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:28:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:28:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:29:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:29:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:30:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:30:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:31:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:31:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:32:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:32:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:33:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:33:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:34:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:34:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:35:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:35:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:36:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:36:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:37:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:37:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:38:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:38:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:39:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:39:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:40:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:40:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:41:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:41:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:42:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:42:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:43:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:43:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:44:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:44:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:45:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:45:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:46:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:46:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:47:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:47:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:48:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:48:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:49:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:49:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:50:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:50:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:51:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:51:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:52:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:52:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:53:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:53:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:54:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:54:54 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:55:24 +0000 Submitted:9 Finished successfully:1
+Progress: time: Wed, 09 Oct 2013 18:55:54 +0000 Submitted:9 Finished successfully:1
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1857-p3i03fi0
+Progress: time: Wed, 09 Oct 2013 18:57:06 +0000
+Progress: time: Wed, 09 Oct 2013 18:57:36 +0000 Submitting:7 Submitted:3
+Progress: time: Wed, 09 Oct 2013 18:58:06 +0000 Submitting:7 Submitted:3
+Progress: time: Wed, 09 Oct 2013 18:58:36 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 18:59:06 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 18:59:36 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:00:06 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:00:36 +0000 Submitted:10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1900-9gnh31e3
+Progress: time: Wed, 09 Oct 2013 19:00:58 +0000
+Failed to secure the socket
+Defective credential detected. Caused by org.globus.gsi.GlobusCredentialException: [JGLOBUS-5] Proxy file (/tmp/x509up_u896092611) not found.
+ at org.globus.gsi.GlobusCredential.<init>(GlobusCredential.java:114)
+ at org.globus.gsi.GlobusCredential.reloadDefaultCredential(GlobusCredential.java:590)
+ at org.globus.gsi.GlobusCredential.getDefaultCredential(GlobusCredential.java:575)
+ at org.globus.gsi.gssapi.GlobusGSSManagerImpl.createCredential(GlobusGSSManagerImpl.java:130)
+ at org.globus.gsi.gssapi.GlobusGSSManagerImpl.createCredential(GlobusGSSManagerImpl.java:71)
+ at org.globus.gsi.gssapi.GlobusGSSManagerImpl.createContext(GlobusGSSManagerImpl.java:287)
+ at org.globus.net.BaseServer.wrapSocket(BaseServer.java:258)
+ at org.globus.net.BaseServer.run(BaseServer.java:240)
+ at java.lang.Thread.run(Thread.java:724)
+Progress: time: Wed, 09 Oct 2013 19:01:28 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:01:58 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:02:28 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:02:58 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:03:28 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:03:58 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:04:28 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:04:58 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:05:28 +0000 Submitted:10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1908-knbbcz8g
+Progress: time: Wed, 09 Oct 2013 19:08:20 +0000
+Progress: time: Wed, 09 Oct 2013 19:08:50 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:09:20 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:09:50 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:10:20 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:10:50 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:11:20 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:11:50 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:12:20 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:12:50 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:13:20 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:13:50 +0000 Submitted:10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1917-grot9cv1
+Progress: time: Wed, 09 Oct 2013 19:17:18 +0000
+Progress: time: Wed, 09 Oct 2013 19:17:48 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:18:18 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:18:48 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:19:18 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:19:48 +0000 Submitted:10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1920-pk9o8dl5
+Progress: time: Wed, 09 Oct 2013 19:20:13 +0000
+Progress: time: Wed, 09 Oct 2013 19:20:43 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:21:13 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:21:43 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:22:13 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 19:22:43 +0000 Submitted:10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 29, shadows variable of same name on line 75
+Warning: Parameter deref, on line 34, shadows variable of same name on line 78
+Warning: Parameter reducer, on line 39, shadows variable of same name on line 77
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1925-lgedrzj1
+Progress: time: Wed, 09 Oct 2013 19:25:35 +0000
+Progress: time: Wed, 09 Oct 2013 19:25:38 +0000 Stage in:1 Submitted:9
+Progress: time: Wed, 09 Oct 2013 19:25:39 +0000 Stage out:1 Finished successfully:9
+Node info for 0 : midway midway261.rcc.local /dev/shm/input_5465.txt /dev/shm/input_19045.txt /dev/shm/input_11168.txt /dev/shm/input_26473.txt /dev/shm/input_19770.txt /dev/shm/input_9302.txt /dev/shm/input_797.txt /dev/shm/input_20119.txt /dev/shm/input_7866.txt /dev/shm/input_7045.txtOn midway
+Final status: Wed, 09 Oct 2013 19:25:39 +0000 Finished successfully:11
+real 7.75
+user 13.38
+sys 2.05
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:10, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 90
+Warning: Parameter deref, on line 34, shadows variable of same name on line 93
+Warning: Parameter deref, on line 49, shadows variable of same name on line 93
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 92
+Could not start execution
+ Compile error in foreach statement at line 73
+ Compile error in procedure invocation at line 75
+ Procedure reduce_data is not declared.
+real 1.77
+user 1.92
+sys 0.10
+chunks 20 filesize 10 site 10 type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:10, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Could not open: 10.xml
+Execution failed:
+
+ Could not find sites file: 10.xml
+
+real 1.93
+user 3.33
+sys 0.15
+chunks 20 filesize 10 site 10 type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1947-e6zv87u6
+Progress: time: Wed, 09 Oct 2013 19:47:56 +0000
+Progress: time: Wed, 09 Oct 2013 19:48:20 +0000 Stage in:1 Submitted:9
+Node info for 0 : midway midway261.rcc.local /dev/shm/input_27482.txt /dev/shm/input_10585.txtNode info for 2 : midway midway427.rcc.local /dev/shm/input_30489.txt /dev/shm/input_6903.txtNode info for 1 : midway midway331.rcc.local /dev/shm/input_28138.txt /dev/shm/input_10951.txt /dev/shm/input_29235.txtNode info for 3 : midway midway428.rcc.local /dev/shm/input_20026.txt /dev/shm/input_16186.txt /dev/shm/input_1772.txtOn midway
+On midway
+On midway
+On midway
+Progress: time: Wed, 09 Oct 2013 19:48:26 +0000 Finished successfully:11
+No events in 10s.
+Finding dependency loops...
+
+Registered futures:
+fileptr[int] comb_out Open, 0 elements, 4 listeners
+----
+int L - F/L:int - Open
+----
+int L - F/L:int - Open
+----
+int L - F/L:int - Open
+----
+int L - F/L:int - Open
+----
+
+Waiting threads:
+Thread: 0-19-3-3-1, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen.swift, line 155
+ assignment, teragen.swift, line 156
+
+Thread: 0-19-1-3-2, waiting on L (declared on line 155)
+ m_comb_data, teragen.swift, line 156
+ m_comb_data, teragen.swift, line 156
+
+Thread: 0-19-2-3-1, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen.swift, line 155
+ assignment, teragen.swift, line 156
+
+Thread: 0-19-0-3-2, waiting on L (declared on line 155)
+ m_comb_data, teragen.swift, line 156
+ m_comb_data, teragen.swift, line 156
+
+Thread: 0-19-2-3-2, waiting on L (declared on line 155)
+ m_comb_data, teragen.swift, line 156
+ m_comb_data, teragen.swift, line 156
+
+Thread: 0-19-3-3-2, waiting on L (declared on line 155)
+ m_comb_data, teragen.swift, line 156
+ m_comb_data, teragen.swift, line 156
+
+Thread: 0-19-1-3-1, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen.swift, line 155
+ assignment, teragen.swift, line 156
+
+Thread: 0-19-0-3-1, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen.swift, line 155
+ assignment, teragen.swift, line 156
+
+----
+No dependency loops found.
+
+The following threads are independently hung:
+Thread: 0-19-3-3-1, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen.swift, line 155
+ assignment, teragen.swift, line 156
+
+Thread: 0-19-1-3-2, waiting on L (declared on line 155)
+ m_comb_data, teragen.swift, line 156
+ m_comb_data, teragen.swift, line 156
+
+Thread: 0-19-2-3-1, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen.swift, line 155
+ assignment, teragen.swift, line 156
+
+Thread: 0-19-0-3-2, waiting on L (declared on line 155)
+ m_comb_data, teragen.swift, line 156
+ m_comb_data, teragen.swift, line 156
+
+Thread: 0-19-2-3-2, waiting on L (declared on line 155)
+ m_comb_data, teragen.swift, line 156
+ m_comb_data, teragen.swift, line 156
+
+Thread: 0-19-1-3-1, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen.swift, line 155
+ assignment, teragen.swift, line 156
+
+Thread: 0-19-3-3-2, waiting on L (declared on line 155)
+ m_comb_data, teragen.swift, line 156
+ m_comb_data, teragen.swift, line 156
+
+Thread: 0-19-0-3-1, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen.swift, line 155
+ assignment, teragen.swift, line 156
+
+----
+
+Irrecoverable error found. Exiting.
+real 41.84
+user 49.12
+sys 10.91
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1949-6qdei6e7
+Progress: time: Wed, 09 Oct 2013 19:49:22 +0000
+Progress: time: Wed, 09 Oct 2013 19:49:27 +0000 Stage in:1 Submitted:9
+Node info for 1 : midway midway331.rcc.local /dev/shm/input_12731.txt /dev/shm/input_6025.txt /dev/shm/input_22774.txt /dev/shm/input_31575.txt /dev/shm/input_14216.txtNode info for 0 : midway midway261.rcc.local /dev/shm/input_28039.txt /dev/shm/input_28537.txt /dev/shm/input_28047.txt /dev/shm/input_18806.txt /dev/shm/input_5023.txtOn midway
+On midway
+Execution failed:
+ The application "mid_bash" is not available for any site/pool in your tc.data catalog
+ m_comb_data, teragen.swift, line 154
+real 7.86
+user 13.92
+sys 2.25
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1950-x6qc5t55
+Progress: time: Wed, 09 Oct 2013 19:50:25 +0000
+Progress: time: Wed, 09 Oct 2013 19:50:33 +0000 Stage in:1 Submitted:9
+Node info for 3 : midway midway428.rcc.local /dev/shm/input_13854.txt /dev/shm/input_18840.txtNode info for 1 : midway midway331.rcc.local /dev/shm/input_21498.txt /dev/shm/input_18007.txt /dev/shm/input_13421.txtNode info for 0 : midway midway261.rcc.local /dev/shm/input_19256.txt /dev/shm/input_10205.txtNode info for 2 : midway midway427.rcc.local /dev/shm/input_31007.txt /dev/shm/input_1042.txt /dev/shm/input_13719.txtOn midway
+On midway
+On midway
+On midway
+Final status: Wed, 09 Oct 2013 19:50:33 +0000 Finished successfully:15
+real 11.69
+user 20.73
+sys 3.71
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:10, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Could not open: 10.xml
+Execution failed:
+
+ Could not find sites file: 10.xml
+
+real 2.60
+user 3.34
+sys 0.17
+chunks 20 filesize 10 site 10 type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-1957-lllsayh7
+Progress: time: Wed, 09 Oct 2013 19:58:01 +0000
+Progress: time: Wed, 09 Oct 2013 19:58:18 +0000 Stage in:1 Submitted:9
+Node info for 0 : midway midway261.rcc.local /dev/shm/input_31856.txt /dev/shm/input_10152.txt /dev/shm/input_32500.txt
+Node info for 2 : midway midway411.rcc.local /dev/shm/input_4176.txt /dev/shm/input_27866.txt
+Node info for 1 : midway midway262.rcc.local /dev/shm/input_14236.txt /dev/shm/input_20678.txt
+Node info for 3 : midway midway412.rcc.local /dev/shm/input_24356.txt /dev/shm/input_21726.txt /dev/shm/input_23161.txt
+On midway
+On midway
+On midway
+On midway
+Final status: Wed, 09 Oct 2013 19:58:18 +0000 Finished successfully:15
+real 19.98
+user 35.37
+sys 7.35
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-2003-raa3ebhd
+Progress: time: Wed, 09 Oct 2013 20:03:18 +0000
+Progress: time: Wed, 09 Oct 2013 20:03:48 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 20:03:49 +0000 Submitted:6 Active:3 Stage out:1
+Node info for 0 : midway midway262.rcc.local /dev/shm/input_1074.txt /dev/shm/input_30403.txt /dev/shm/input_14938.txt
+Node info for 2 : midway midway412.rcc.local /dev/shm/input_5420.txt /dev/shm/input_4267.txt /dev/shm/input_24826.txt
+Node info for 3 : midway midway413.rcc.local /dev/shm/input_20617.txt /dev/shm/input_4115.txt
+Node info for 1 : midway midway411.rcc.local /dev/shm/input_31499.txt /dev/shm/input_24425.txt
+On midway
+On midway
+On midway
+On midway
+Final status: Wed, 09 Oct 2013 20:03:49 +0000 Finished successfully:15
+real 35.17
+user 60.98
+sys 13.38
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-2007-0tm3xccc
+Progress: time: Wed, 09 Oct 2013 20:07:52 +0000
+Progress: time: Wed, 09 Oct 2013 20:08:14 +0000 Stage in:1 Submitted:9
+Node info for 2 : midway midway412.rcc.local /dev/shm/input_6821.txt /dev/shm/input_8423.txt
+Node info for 0 : midway midway262.rcc.local /dev/shm/input_7801.txt /dev/shm/input_32300.txt /dev/shm/input_29069.txt
+Node info for 1 : midway midway411.rcc.local /dev/shm/input_27883.txt /dev/shm/input_11835.txt /dev/shm/input_21220.txt
+Node info for 3 : midway midway413.rcc.local /dev/shm/input_24416.txt /dev/shm/input_12377.txt
+On midway
+On midway
+On midway
+On midway
+Final status: Wed, 09 Oct 2013 20:08:14 +0000 Finished successfully:15
+real 26.51
+user 45.05
+sys 10.12
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:10, CHUNKS:20 X FILESIZE:10
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Could not open: 10.xml
+Execution failed:
+
+ Could not find sites file: 10.xml
+
+real 2.58
+user 3.36
+sys 0.13
+chunks 20 filesize 10 site 10 type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-2019-bdopd3h3
+Progress: time: Wed, 09 Oct 2013 20:19:05 +0000
+Progress: time: Wed, 09 Oct 2013 20:19:20 +0000 Stage in:1 Submitted:9
+Node info for 1 : midway midway034.rcc.local /dev/shm/input_10078.txt /dev/shm/input_26041.txt /dev/shm/input_14557.txt
+Node info for 2 : midway midway035.rcc.local /dev/shm/input_10768.txt /dev/shm/input_14250.txt
+Node info for 0 : midway midway033.rcc.local /dev/shm/input_32388.txt /dev/shm/input_30737.txt /dev/shm/input_10739.txt
+Node info for 3 : midway midway202.rcc.local /dev/shm/input_17117.txt /dev/shm/input_28339.txt
+On midway
+On midway
+On midway
+On midway
+ Length of array : 4 len-1 mod K_way: 0
+ Length of array : 0 len-1 mod K_way: 0
+Execution failed:
+ org.griphyn.vdl.mapping.InvalidPathException: Array index '0' not found for bucket of size 0 comb0001.0000.out going to bucket:0
+ comb0001.0003.out going to bucket:0
+ comb0001.0002.out going to bucket:0
+ comb0001.0001.out going to bucket:0
+bucket[0] contains 4 items
+
+ b_reduce_data, teragen.swift, line 75
+ b_reduce_data, teragen.swift, line 75
+ b_kway, teragen.swift, line 164
+real 18.63
+user 34.15
+sys 6.89
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-2045-6ovixk1d
+Progress: time: Wed, 09 Oct 2013 20:45:07 +0000
+Progress: time: Wed, 09 Oct 2013 20:45:21 +0000 Stage in:1 Submitted:9
+Final status: Wed, 09 Oct 2013 20:45:22 +0000 Finished successfully:12
+real 19.78
+user 42.81
+sys 6.12
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Could not start execution
+ Compile error in assignment at line 138: You cannot assign value of type int[int] to a variable of type int
+
+real 1.12
+user 1.99
+sys 0.11
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Could not start execution
+ Compile error in assignment at line 138: You cannot assign value of type int[int] to a variable of type int
+
+real 2.57
+user 2.00
+sys 0.13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Could not start execution
+ Compile error in foreach statement at line 141
+ Compile error in procedure invocation at line 143
+ Wrong type for parameter number 0, expected file, got file[int]
+real 1.16
+user 1.98
+sys 0.10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-2058-5sej2xl2
+Progress: time: Wed, 09 Oct 2013 20:58:23 +0000
+site : 0
+site : 1
+Progress: time: Wed, 09 Oct 2013 20:58:52 +0000 Stage in:1 Submitted:9
+Final status: Wed, 09 Oct 2013 20:58:52 +0000 Finished successfully:12
+real 32.29
+user 80.28
+sys 12.05
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-2103-47zropma
+Progress: time: Wed, 09 Oct 2013 21:03:35 +0000
+Execution failed:
+ tracef(): %i requires an int! Variable "sites" is a string
+ swiftscript:tracef, teragen.swift, line 145
+real 2.49
+user 4.38
+sys 0.17
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-2103-5naohne8
+Progress: time: Wed, 09 Oct 2013 21:03:54 +0000
+site : beagle
+site : midway
+Progress: time: Wed, 09 Oct 2013 21:04:24 +0000 Submitted:10
+Final status: Wed, 09 Oct 2013 21:04:25 +0000 Finished successfully:12
+real 33.58
+user 80.63
+sys 13.55
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Could not start execution
+ Compile error in foreach statement at line 146
+ Compile error in foreach statement at line 152
+ Compile error in procedure invocation at line 153
+ Variable i was not declared in this scope.
+real 1.16
+user 2.13
+sys 0.10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Could not start execution
+ Compile error in foreach statement at line 146
+ Compile error in foreach statement at line 153
+ Compile error in procedure invocation at line 154
+ Variable i was not declared in this scope.
+real 1.14
+user 2.01
+sys 0.10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-2144-07sqijid
+Progress: time: Wed, 09 Oct 2013 21:44:20 +0000
+site : midway
+site : beagle
+Progress: time: Wed, 09 Oct 2013 21:44:49 +0000 Stage in:1 Submitted:9
+Execution failed:
+ java.io.FileNotFoundException: node.out (No such file or directory)
+ swiftscript:readData, teragen.swift, line 149
+ assignment, teragen.swift, line 151
+real 31.67
+user 67.53
+sys 7.53
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131009-2145-rziqoj83
+Progress: time: Wed, 09 Oct 2013 21:45:56 +0000
+site : midway
+site : beagle
+Progress: time: Wed, 09 Oct 2013 21:46:26 +0000 Submitted:10
+Progress: time: Wed, 09 Oct 2013 21:46:31 +0000 Stage in:1 Submitted:9
+Node info for midway : midway midway213.rcc.local /dev/shm/input_21777.txt /dev/shm/input_9764.txt
+Node info for midway : midway midway096.rcc.local /dev/shm/input_2591.txt /dev/shm/input_125.txt /dev/shm/input_9238.txt
+Node info for midway : midway midway070.rcc.local /dev/shm/input_119.txt /dev/shm/input_19489.txt
+Node info for midway : midway midway261.rcc.local /dev/shm/input_32043.txt /dev/shm/input_18849.txt /dev/shm/input_22180.txt
+Final status: Wed, 09 Oct 2013 21:46:31 +0000 Finished successfully:16
+real 38.07
+user 87.73
+sys 15.97
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Could not start execution
+ Compile error in foreach statement at line 146
+ Compile error in procedure invocation at line 159
+ Variable b_reducer was not declared in this scope.
+real 2.41
+user 2.13
+sys 0.13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Could not start execution
+ Compile error in foreach statement at line 146
+ Compile error in procedure invocation at line 159
+ Variable b_reducer was not declared in this scope.
+real 1.17
+user 2.15
+sys 0.10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Could not start execution
+ Compile error in foreach statement at line 146
+ Compile error in procedure invocation at line 159
+ Variable b_reducer was not declared in this scope.
+real 1.17
+user 2.14
+sys 0.11
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1517-jpo9t3c2
+Progress: time: Thu, 10 Oct 2013 15:17:44 +0000
+site : midway
+site : beagle
+Progress: time: Thu, 10 Oct 2013 15:17:56 +0000 Stage in:1 Submitted:9
+Node info for midway : midway midway018.rcc.local /dev/shm/input_31349.txt /dev/shm/input_7767.txt
+Node info for midway : midway midway007.rcc.local /dev/shm/input_20213.txt /dev/shm/input_2825.txt /dev/shm/input_3550.txt
+Node info for midway : midway midway008.rcc.local /dev/shm/input_26275.txt /dev/shm/input_22708.txt
+Node info for midway : midway midway016.rcc.local /dev/shm/input_24920.txt /dev/shm/input_21557.txt /dev/shm/input_20179.txt
+Progress: time: Thu, 10 Oct 2013 15:18:14 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 15:18:44 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 15:19:14 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 15:19:44 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 15:20:14 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 15:20:44 +0000 Finished successfully:16
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1520-e45o4s88
+Progress: time: Thu, 10 Oct 2013 15:21:00 +0000
+site : beagle
+site : midway
+Progress: time: Thu, 10 Oct 2013 15:21:30 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:22:00 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:22:30 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:23:00 +0000 Submitted:10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1523-h5a4abf9
+Progress: time: Thu, 10 Oct 2013 15:23:58 +0000
+site : beagle
+site : midway
+Progress: time: Thu, 10 Oct 2013 15:24:10 +0000 Stage in:1 Submitting:9
+Progress: time: Thu, 10 Oct 2013 15:24:28 +0000 Submitted:8 Finished successfully:2
+Progress: time: Thu, 10 Oct 2013 15:24:58 +0000 Submitted:8 Finished successfully:2
+Progress: time: Thu, 10 Oct 2013 15:25:28 +0000 Submitted:8 Finished successfully:2
+Progress: time: Thu, 10 Oct 2013 15:25:58 +0000 Submitted:8 Finished successfully:2
+Progress: time: Thu, 10 Oct 2013 15:26:28 +0000 Submitted:8 Finished successfully:2
+Progress: time: Thu, 10 Oct 2013 15:26:58 +0000 Submitted:8 Finished successfully:2
+Progress: time: Thu, 10 Oct 2013 15:27:28 +0000 Submitted:8 Finished successfully:2
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1528-q8aknhdf
+Progress: time: Thu, 10 Oct 2013 15:28:45 +0000
+site : midway
+site : beagle
+Progress: time: Thu, 10 Oct 2013 15:28:57 +0000 Submitting:9 Submitted:1
+Progress: time: Thu, 10 Oct 2013 15:29:15 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:29:45 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:30:15 +0000 Submitted:10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1530-seqj9d65
+Progress: time: Thu, 10 Oct 2013 15:30:35 +0000
+site : beagle
+site : midway
+Progress: time: Thu, 10 Oct 2013 15:30:47 +0000 Submitting:9 Submitted:1
+Progress: time: Thu, 10 Oct 2013 15:31:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:31:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:32:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:32:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:33:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:33:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:34:05 +0000 Submitted:10
+Channels: {null at id://u-1ecc03a5-141a2fdd92e--8000-u1ae06ffb-141a2fdd95d--8000S=MetaChannel [context: service-63974, boundTo: GSSChannel [type: service, contact: 1466857485, id: 2, context: service-63974]], /C=US/O=JavaCoG/OU=AutoCA/CN=User at https://192.5.86.107:60000=MetaChannel [context: service-63974, boundTo: GSSChannel [type: service, contact: 1466857485, id: 2, context: service-63974]], null at id://u60e0cef1-141a2fc2cb2--8000-u1da9abbc-141a2fc2cc6--8000S=MetaChannel [context: service-60981, boundTo: BufferingChannel]}
+Context: service-63866
+Meta context: service-60981
+Channels: {null at id://u-1ecc03a5-141a2fdd92e--8000-u1ae06ffb-141a2fdd95d--8000S=MetaChannel [context: service-63974, boundTo: GSSChannel [type: service, contact: 1466857485, id: 2, context: service-63974]], /C=US/O=JavaCoG/OU=AutoCA/CN=User at https://192.5.86.107:60000=MetaChannel [context: service-63974, boundTo: GSSChannel [type: service, contact: 1466857485, id: 2, context: service-63974]], null at id://u60e0cef1-141a2fc2cb2--8000-u1da9abbc-141a2fc2cc6--8000S=MetaChannel [context: service-60981, boundTo: BufferingChannel]}
+Context: service-63880
+Meta context: service-60981
+Channels: {null at id://u-1ecc03a5-141a2fdd92e--8000-u1ae06ffb-141a2fdd95d--8000S=MetaChannel [context: service-63974, boundTo: GSSChannel [type: service, contact: 1466857485, id: 2, context: service-63974]], /C=US/O=JavaCoG/OU=AutoCA/CN=User at https://192.5.86.107:60000=MetaChannel [context: service-63974, boundTo: GSSChannel [type: service, contact: 1466857485, id: 2, context: service-63974]], null at id://u60e0cef1-141a2fc2cb2--8000-u1da9abbc-141a2fc2cc6--8000S=MetaChannel [context: service-60981, boundTo: BufferingChannel]}
+Context: service-60627
+Meta context: service-60981
+Channels: {null at id://u-1ecc03a5-141a2fdd92e--8000-u1ae06ffb-141a2fdd95d--8000S=MetaChannel [context: service-63974, boundTo: GSSChannel [type: service, contact: 1466857485, id: 2, context: service-63974]], /C=US/O=JavaCoG/OU=AutoCA/CN=User at https://192.5.86.107:60000=MetaChannel [context: service-63974, boundTo: GSSChannel [type: service, contact: 1466857485, id: 2, context: service-63974]], null at id://u60e0cef1-141a2fc2cb2--8000-u1da9abbc-141a2fc2cc6--8000S=MetaChannel [context: service-60981, boundTo: BufferingChannel]}
+Context: service-64302
+Meta context: service-60981
+Progress: time: Thu, 10 Oct 2013 15:34:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:35:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:35:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:36:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:36:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:37:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:37:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:38:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:38:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:39:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:39:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:40:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:40:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:41:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:41:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:42:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:42:35 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:43:05 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 15:43:35 +0000 Submitted:10
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1545-qv7kvbr3
+Progress: time: Thu, 10 Oct 2013 15:45:06 +0000
+site : beagle
+site : midway
+Progress: time: Thu, 10 Oct 2013 15:45:17 +0000 Submitting:9 Submitted:1
+Progress: time: Thu, 10 Oct 2013 15:45:24 +0000 Stage in:1 Submitted:9
+Progress: time: Thu, 10 Oct 2013 15:45:26 +0000 Stage in:1 Active:1 Finished successfully:10
+Node info for beagle : beagle nid00264 /dev/shm/input_20601.txt /dev/shm/input_23959.txt /dev/shm/input_32269.txt /dev/shm/input_30891.txt /dev/shm/input_6252.txt /dev/shm/input_10222.txt /dev/shm/input_7403.txt /dev/shm/input_18853.txt /dev/shm/input_20826.txt /dev/shm/input_5630.txt
+Progress: time: Thu, 10 Oct 2013 15:45:36 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:46:06 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:46:36 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:47:06 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:47:36 +0000 Finished successfully:13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1551-pdzj1x66
+Progress: time: Thu, 10 Oct 2013 15:51:16 +0000
+site : midway
+site : beagle
+Progress: time: Thu, 10 Oct 2013 15:51:28 +0000 Submitting:9 Submitted:1
+Progress: time: Thu, 10 Oct 2013 15:51:37 +0000 Stage in:1 Submitted:9
+Progress: time: Thu, 10 Oct 2013 15:51:39 +0000 Stage in:1 Active:1 Finished successfully:10
+Node info for beagle : beagle nid00264 /dev/shm/input_22363.txt /dev/shm/input_31826.txt /dev/shm/input_7995.txt /dev/shm/input_14777.txt /dev/shm/input_20971.txt /dev/shm/input_9319.txt /dev/shm/input_8379.txt /dev/shm/input_26691.txt /dev/shm/input_15132.txt /dev/shm/input_5433.txt
+Progress: time: Thu, 10 Oct 2013 15:51:41 +0000 Stage out:1 Finished successfully:12
+Progress: time: Thu, 10 Oct 2013 15:51:46 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:52:16 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:52:46 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:53:16 +0000 Finished successfully:13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1553-q96017f9
+Progress: time: Thu, 10 Oct 2013 15:53:28 +0000
+site : beagle
+site : midway
+Progress: time: Thu, 10 Oct 2013 15:53:40 +0000 Submitting:9 Submitted:1
+Progress: time: Thu, 10 Oct 2013 15:53:49 +0000 Stage in:1 Submitted:9
+Progress: time: Thu, 10 Oct 2013 15:53:50 +0000 Stage in:1 Active:1 Finished successfully:10
+Number of nodes on beagle is 1
+Node info for beagle : beagle nid00264 /dev/shm/input_5224.txt /dev/shm/input_21128.txt /dev/shm/input_10949.txt /dev/shm/input_22294.txt /dev/shm/input_21237.txt /dev/shm/input_20497.txt /dev/shm/input_16185.txt /dev/shm/input_24711.txt /dev/shm/input_14104.txt /dev/shm/input_16399.txt
+Progress: time: Thu, 10 Oct 2013 15:53:58 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:54:28 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:54:58 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:55:28 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:55:58 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:56:28 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:56:58 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:57:28 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:57:58 +0000 Finished successfully:13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 144, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 145, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1558-v57t8rq3
+Progress: time: Thu, 10 Oct 2013 15:58:22 +0000
+site : midway
+site : beagle
+Progress: time: Thu, 10 Oct 2013 15:58:35 +0000 Submitting:9 Submitted:1
+Progress: time: Thu, 10 Oct 2013 15:58:40 +0000 Stage in:1 Submitted:9
+Progress: time: Thu, 10 Oct 2013 15:58:41 +0000 Stage in:1 Active:1 Finished successfully:10
+Number of nodes on beagle is 1
+Number of nodes on midway is 0
+Node info for beagle : beagle nid00264 /dev/shm/input_5719.txt /dev/shm/input_20350.txt /dev/shm/input_19655.txt /dev/shm/input_16243.txt /dev/shm/input_22767.txt /dev/shm/input_20438.txt /dev/shm/input_27961.txt /dev/shm/input_11757.txt /dev/shm/input_5048.txt /dev/shm/input_1823.txt
+Progress: time: Thu, 10 Oct 2013 15:58:52 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:59:22 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 15:59:52 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:00:22 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:00:52 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:01:22 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:01:52 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:02:22 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:02:52 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:03:22 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:03:52 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:04:22 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:04:52 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:05:22 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:05:52 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:06:22 +0000 Finished successfully:13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 144, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 145, might have multiple conflicting writers
+Could not start execution
+ Compile error in foreach statement at line 146
+ Compile error in procedure invocation at line 164
+ Wrong type for parameter number 0, expected file, got fileptr
+real 1.20
+user 2.20
+sys 0.09
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 144, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 145, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1611-5x7vuny7
+Progress: time: Thu, 10 Oct 2013 16:11:47 +0000
+Progress: time: Thu, 10 Oct 2013 16:11:58 +0000 Submitting:9 Submitted:1
+Progress: time: Thu, 10 Oct 2013 16:12:10 +0000 Stage in:1 Submitted:9
+Progress: time: Thu, 10 Oct 2013 16:12:12 +0000 Stage in:1 Active:1 Finished successfully:10
+Number of nodes on midway is 0
+Number of nodes on beagle is 1
+Nothing to do on midway
+Node info for beagle : beagle nid00264 /dev/shm/input_14.txt /dev/shm/input_24426.txt /dev/shm/input_22174.txt /dev/shm/input_488.txt /dev/shm/input_21141.txt /dev/shm/input_7782.txt /dev/shm/input_5274.txt /dev/shm/input_14183.txt /dev/shm/input_18063.txt /dev/shm/input_5299.txt
+Progress: time: Thu, 10 Oct 2013 16:12:14 +0000 Stage out:1 Finished successfully:12
+Progress: time: Thu, 10 Oct 2013 16:12:17 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:12:47 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:13:17 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:13:47 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:14:17 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:14:47 +0000 Finished successfully:13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 144, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 145, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1615-o37xxie1
+Progress: time: Thu, 10 Oct 2013 16:15:03 +0000
+Progress: time: Thu, 10 Oct 2013 16:15:33 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 16:15:35 +0000 Stage in:1 Submitted:9
+Number of nodes on beagle is 0
+Nothing to do on beagle
+Number of nodes on midway is 4
+Node info for midway : midway midway016.rcc.local /dev/shm/input_11651.txt /dev/shm/input_7933.txt /dev/shm/input_23631.txt
+Node info for midway : midway midway028.rcc.local /dev/shm/input_3241.txt /dev/shm/input_45.txt
+Node info for midway : midway midway029.rcc.local /dev/shm/input_9424.txt /dev/shm/input_8913.txt
+Node info for midway : midway midway018.rcc.local /dev/shm/input_18988.txt /dev/shm/input_23352.txt /dev/shm/input_9728.txt
+Progress: time: Thu, 10 Oct 2013 16:16:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:16:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:17:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:17:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:18:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:18:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:19:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:19:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:20:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:20:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:21:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:21:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:22:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:22:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:23:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:23:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:24:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:24:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:25:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:25:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:26:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:26:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:27:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:27:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:28:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:28:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:29:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:29:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:30:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:30:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:31:03 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:31:33 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 16:32:03 +0000 Finished successfully:16
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 144, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 145, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1645-enhm7fo0
+Progress: time: Thu, 10 Oct 2013 16:45:21 +0000
+Progress: time: Thu, 10 Oct 2013 16:45:45 +0000 Stage in:1 Submitted:9
+Number of nodes on beagle is 0
+Nothing to do on beagle
+Number of nodes on midway is 4
+Node info for midway : midway midway018.rcc.local /dev/shm/input_14224.txt /dev/shm/input_25618.txt
+Node info for midway : midway midway016.rcc.local /dev/shm/input_16970.txt /dev/shm/input_25671.txt /dev/shm/input_8917.txt
+Node info for midway : midway midway005.rcc.local /dev/shm/input_24352.txt /dev/shm/input_19042.txt
+Node info for midway : midway midway028.rcc.local /dev/shm/input_26215.txt /dev/shm/input_27017.txt /dev/shm/input_2626.txt
+Progress: time: Thu, 10 Oct 2013 16:45:51 +0000 Finished successfully:16
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Could not compile SwiftScript source: line 169:9: unexpected token: fileptr
+real 0.47
+user 0.60
+sys 0.05
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Could not compile SwiftScript source: line 169:9: expecting '}', found '{'
+real 0.47
+user 0.62
+sys 0.05
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Could not compile SwiftScript source: line 169:9: unexpected token: fileptr
+real 0.47
+user 0.59
+sys 0.07
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Could not compile SwiftScript source: line 169:9: expecting '}', found 'type'
+real 0.47
+user 0.60
+sys 0.05
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 146, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 147, might have multiple conflicting writers
+Warning: Variable m_comb_out, defined on line 139, might have multiple conflicting writers
+Warning: Variable m_comb_err, defined on line 140, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1652-a7f2zqid
+Progress: time: Thu, 10 Oct 2013 16:52:44 +0000
+Progress: time: Thu, 10 Oct 2013 16:52:59 +0000 Stage in:1 Submitted:9
+Number of nodes on beagle is 0
+Nothing to do on beagle
+Number of nodes on midway is 4
+Node info for midway : midway midway016.rcc.local /dev/shm/input_10688.txt /dev/shm/input_1421.txt /dev/shm/input_23026.txt
+Node info for midway : midway midway005.rcc.local /dev/shm/input_5569.txt /dev/shm/input_1743.txt
+Node info for midway : midway midway018.rcc.local /dev/shm/input_17384.txt /dev/shm/input_23640.txt
+Node info for midway : midway midway007.rcc.local /dev/shm/input_2173.txt /dev/shm/input_22103.txt /dev/shm/input_11988.txt
+Execution failed:
+ java.lang.RuntimeException: getArrayValue called on a non-array: m_comb_out[1]:fileptr - Closed
+real 17.97
+user 43.06
+sys 5.88
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 146, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 147, might have multiple conflicting writers
+Warning: Variable m_comb_out, defined on line 139, might have multiple conflicting writers
+Warning: Variable m_comb_err, defined on line 140, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1653-4xucixof
+Progress: time: Thu, 10 Oct 2013 16:53:28 +0000
+Progress: time: Thu, 10 Oct 2013 16:53:32 +0000 Stage in:1 Submitted:9
+Number of nodes on midway is 1
+Node info for midway : midway midway005.rcc.local /dev/shm/input_28317.txt /dev/shm/input_30799.txt /dev/shm/input_20303.txt /dev/shm/input_6115.txt /dev/shm/input_9237.txt /dev/shm/input_21367.txt /dev/shm/input_3006.txt /dev/shm/input_3571.txt /dev/shm/input_5382.txt /dev/shm/input_11365.txt
+Number of nodes on beagle is 0
+Nothing to do on beagle
+Progress: time: Thu, 10 Oct 2013 16:53:58 +0000 Finished successfully:13
+Progress: time: Thu, 10 Oct 2013 16:54:28 +0000 Finished successfully:13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1655-i7oale66
+Progress: time: Thu, 10 Oct 2013 16:55:37 +0000
+Progress: time: Thu, 10 Oct 2013 16:55:46 +0000 Stage in:1 Submitted:9
+Number of nodes on beagle is 0
+Nothing to do on beagle
+Number of nodes on midway is 4
+Node info for midway : midway midway005.rcc.local /dev/shm/input_9123.txt /dev/shm/input_1435.txt /dev/shm/input_12723.txt
+Node info for midway : midway midway018.rcc.local /dev/shm/input_18117.txt /dev/shm/input_17657.txt
+Node info for midway : midway midway016.rcc.local /dev/shm/input_13793.txt /dev/shm/input_14731.txt /dev/shm/input_16387.txt
+Node info for midway : midway midway007.rcc.local /dev/shm/input_7022.txt /dev/shm/input_11030.txt
+Progress: time: Thu, 10 Oct 2013 16:56:07 +0000 Finished successfully:16
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-1913-8qpobjrf
+Warning: The @ syntax for function invocation is deprecated
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Progress: time: Thu, 10 Oct 2013 19:13:36 +0000
+Progress: time: Thu, 10 Oct 2013 19:13:37 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 19:13:38 +0000 Submitted:6 Active:1 Finished successfully:3
+Number of nodes on midway is 1
+Node info for midway : midway midway010.rcc.local /dev/shm/input_25462.txt /dev/shm/input_27202.txt /dev/shm/input_30524.txt /dev/shm/input_19840.txt /dev/shm/input_14619.txt /dev/shm/input_9753.txt /dev/shm/input_31623.txt /dev/shm/input_23380.txt /dev/shm/input_14619.txt /dev/shm/input_7770.txt
+Number of nodes on beagle is 0
+Nothing to do on beagle
+Progress: time: Thu, 10 Oct 2013 19:13:39 +0000 Finished successfully:13
+No events in 1s.
+Finding dependency loops...
+
+Waiting threads:
+Thread: R-20-0-8-1-2x3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 518
+
+Thread: R-20-0-8-1-2-1, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 477
+
+Thread: R-20-0-8-1x2-2, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 1038
+
+Thread: R-20-0-8-1-2-0, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 471
+
+Thread: R-20-0-8-1-2-3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 524
+
+Thread: R-20-0-8-1-2-4, waiting on bucket_out (declared on line 101)
+ swiftscript:length, teragen, line 552
+
+Thread: R-20-0-8-1-3, waiting on site_ptr[1] (declared on line 150)
+ swift:stageIn, teragen, line 589
+ swift:execute, teragen, line 586
+ m_dereference, teragen, line 1051
+
+----
+No dependency loops found.
+
+The following threads are independently hung:
+Thread: R-20-0-8-1-2x3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 518
+
+Thread: R-20-0-8-1-2-1, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 477
+
+Thread: R-20-0-8-1x2-2, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 1038
+
+Thread: R-20-0-8-1-2-0, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 471
+
+Thread: R-20-0-8-1-2-3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 524
+
+Thread: R-20-0-8-1-2-4, waiting on bucket_out (declared on line 101)
+ swiftscript:length, teragen, line 552
+
+Thread: R-20-0-8-1-3, waiting on site_ptr[1] (declared on line 150)
+ swift:stageIn, teragen, line 589
+ swift:execute, teragen, line 586
+ m_dereference, teragen, line 1051
+
+----
+
+Irrecoverable error found. Exiting.
+real 8.27
+user 4.33
+sys 0.26
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-1915-1culqi4c
+Warning: The @ syntax for function invocation is deprecated
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Progress: time: Thu, 10 Oct 2013 19:15:11 +0000
+Progress: time: Thu, 10 Oct 2013 19:15:12 +0000 Submitted:10
+Number of nodes on beagle is 0
+Nothing to do on beagle
+Number of nodes on midway is 4
+Node info for midway : midway midway010.rcc.local /dev/shm/input_26482.txt /dev/shm/input_15295.txt /dev/shm/input_24289.txt /dev/shm/input_25451.txt /dev/shm/input_6909.txt /dev/shm/input_13657.txt /dev/shm/input_10102.txt
+Node info for midway : midway midway011.rcc.local /dev/shm/input_27634.txt
+Node info for midway : midway midway012.rcc.local /dev/shm/input_8011.txt
+Node info for midway : midway midway013.rcc.local /dev/shm/input_24090.txt
+Progress: time: Thu, 10 Oct 2013 19:15:38 +0000 Submitted:4 Finished successfully:12
+Progress: time: Thu, 10 Oct 2013 19:15:39 +0000 Finished successfully:16
+No events in 1s.
+Finding dependency loops...
+
+Waiting threads:
+Thread: R-20-1-8-1-2-1, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 219
+
+Thread: R-20-1-8-1-3, waiting on site_ptr[1] (declared on line 150)
+ swift:stageIn, teragen, line 560
+ swift:execute, teragen, line 557
+ m_dereference, teragen, line 1051
+
+Thread: R-20-1-8-1-2-0, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 213
+
+Thread: R-20-1-8-1-2-4, waiting on bucket_out (declared on line 101)
+ swiftscript:length, teragen, line 294
+
+Thread: R-20-1-8-1-2-3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 266
+
+Thread: R-20-1-8-1x2-2, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 1038
+
+Thread: R-20-1-8-1-2x3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 260
+
+----
+No dependency loops found.
+
+The following threads are independently hung:
+Thread: R-20-1-8-1-2-1, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 219
+
+Thread: R-20-1-8-1-3, waiting on site_ptr[1] (declared on line 150)
+ swift:stageIn, teragen, line 560
+ swift:execute, teragen, line 557
+ m_dereference, teragen, line 1051
+
+Thread: R-20-1-8-1-2-0, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 213
+
+Thread: R-20-1-8-1-2-3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 266
+
+Thread: R-20-1-8-1-2-4, waiting on bucket_out (declared on line 101)
+ swiftscript:length, teragen, line 294
+
+Thread: R-20-1-8-1-2x3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 260
+
+Thread: R-20-1-8-1x2-2, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 1038
+
+----
+
+Irrecoverable error found. Exiting.
+real 31.34
+user 4.58
+sys 0.24
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-1916-p5yuapcg
+Warning: The @ syntax for function invocation is deprecated
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Progress: time: Thu, 10 Oct 2013 19:16:25 +0000
+Progress: time: Thu, 10 Oct 2013 19:16:26 +0000 Submitted:10
+Nothing to do on beagle
+Number of nodes on beagle is 0
+Number of nodes on midway is 1
+Node info for midway : midway midway010.rcc.local /dev/shm/input_29258.txt /dev/shm/input_17272.txt /dev/shm/input_1933.txt /dev/shm/input_12641.txt /dev/shm/input_27273.txt /dev/shm/input_27971.txt /dev/shm/input_1230.txt /dev/shm/input_13476.txt /dev/shm/input_5166.txt /dev/shm/input_12313.txt
+Progress: time: Thu, 10 Oct 2013 19:16:27 +0000 Submitted:1 Finished successfully:12
+Progress: time: Thu, 10 Oct 2013 19:16:28 +0000 Finished successfully:13
+No events in 1s.
+Finding dependency loops...
+
+Waiting threads:
+Thread: R-20-0-8-1-2x3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 260
+
+Thread: R-20-0-8-1-2-3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 266
+
+Thread: R-20-0-8-1-2-0, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 213
+
+Thread: R-20-0-8-1-2-4, waiting on bucket_out (declared on line 101)
+ swiftscript:length, teragen, line 294
+
+Thread: R-20-0-8-1-3, waiting on site_ptr[1] (declared on line 150)
+ swift:stageIn, teragen, line 560
+ swift:execute, teragen, line 557
+ m_dereference, teragen, line 1051
+
+Thread: R-20-0-8-1-2-1, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 219
+
+Thread: R-20-0-8-1x2-2, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 1038
+
+----
+No dependency loops found.
+
+The following threads are independently hung:
+Thread: R-20-0-8-1-2x3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 260
+
+Thread: R-20-0-8-1-2-3, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 266
+
+Thread: R-20-0-8-1-2-4, waiting on bucket_out (declared on line 101)
+ swiftscript:length, teragen, line 294
+
+Thread: R-20-0-8-1-2-0, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 213
+
+Thread: R-20-0-8-1-3, waiting on site_ptr[1] (declared on line 150)
+ swift:stageIn, teragen, line 560
+ swift:execute, teragen, line 557
+ m_dereference, teragen, line 1051
+
+Thread: R-20-0-8-1-2-1, waiting on len (declared on line 89)
+ swiftop:subtraction, teragen, line 219
+
+Thread: R-20-0-8-1x2-2, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 1038
+
+----
+
+Irrecoverable error found. Exiting.
+real 6.28
+user 4.39
+sys 0.22
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-1920-kcirnkr6
+Warning: The @ syntax for function invocation is deprecated
+Warning: Variable comb_out, defined on line 143, might have multiple conflicting writers
+Warning: Variable comb_err, defined on line 144, might have multiple conflicting writers
+Progress: time: Thu, 10 Oct 2013 19:20:48 +0000
+Progress: time: Thu, 10 Oct 2013 19:20:49 +0000 Submitted:10
+Nothing to do on beagle
+Number of nodes on beagle is 0
+Number of nodes on midway is 4
+Node info for midway : midway midway010.rcc.local /dev/shm/input_20054.txt /dev/shm/input_23118.txt /dev/shm/input_23841.txt
+Node info for midway : midway midway012.rcc.local /dev/shm/input_29497.txt /dev/shm/input_11492.txt /dev/shm/input_9829.txt
+Node info for midway : midway midway013.rcc.local /dev/shm/input_26182.txt /dev/shm/input_15674.txt
+Node info for midway : midway midway011.rcc.local /dev/shm/input_5639.txt /dev/shm/input_4107.txt
+Progress: time: Thu, 10 Oct 2013 19:21:08 +0000 Stage out:2 Finished successfully:14
+Progress: time: Thu, 10 Oct 2013 19:21:09 +0000 Finished successfully:16
+No events in 1s.
+Finding dependency loops...
+
+Waiting threads:
+Thread: R-20-0-8-1x2-2, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 1021
+
+----
+No dependency loops found.
+
+The following threads are independently hung:
+Thread: R-20-0-8-1x2-2, waiting on comb_out[1] (declared on line 143)
+ swiftscript:length, teragen, line 1021
+
+----
+
+Irrecoverable error found. Exiting.
+real 24.30
+user 4.36
+sys 0.28
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-1929-9592dzya
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Thu, 10 Oct 2013 19:29:27 +0000
+Progress: time: Thu, 10 Oct 2013 19:29:28 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 19:29:36 +0000 Stage in:2 Submitted:8
+Nothing to do on beagle
+Number of nodes on beagle is 0
+Number of nodes on midway is 4
+Node info for midway : midway midway009.rcc.local /dev/shm/input_2766.txt /dev/shm/input_9868.txt
+Node info for midway : midway midway010.rcc.local /dev/shm/input_20283.txt /dev/shm/input_29910.txt /dev/shm/input_11914.txt
+Node info for midway : midway midway011.rcc.local /dev/shm/input_21856.txt /dev/shm/input_16435.txt
+Node info for midway : midway midway012.rcc.local /dev/shm/input_9334.txt /dev/shm/input_22716.txt /dev/shm/input_5826.txt
+Progress: time: Thu, 10 Oct 2013 19:29:37 +0000 Finished successfully:16
+No events in 1s.
+Finding dependency loops...
+
+Waiting threads:
+Thread: R-16-1-12-1x2-2, waiting on comb_out (declared on line 154)
+ swiftscript:length, teragen, line 1025
+
+----
+No dependency loops found.
+
+The following threads are independently hung:
+Thread: R-16-1-12-1x2-2, waiting on comb_out (declared on line 154)
+ swiftscript:length, teragen, line 1025
+
+----
+
+Irrecoverable error found. Exiting.
+real 16.09
+user 4.49
+sys 0.30
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1930-2719pj69
+Progress: time: Thu, 10 Oct 2013 19:30:29 +0000
+Progress: time: Thu, 10 Oct 2013 19:30:38 +0000 Stage in:1 Submitted:9
+Number of nodes on beagle is 0
+Nothing to do on beagle
+Number of nodes on midway is 4
+Node info for midway : midway midway011.rcc.local /dev/shm/input_31627.txt /dev/shm/input_4701.txt /dev/shm/input_16602.txt
+Node info for midway : midway midway010.rcc.local /dev/shm/input_1688.txt /dev/shm/input_4208.txt /dev/shm/input_8293.txt
+Node info for midway : midway midway009.rcc.local /dev/shm/input_6033.txt /dev/shm/input_7190.txt
+Node info for midway : midway midway012.rcc.local /dev/shm/input_10608.txt /dev/shm/input_32741.txt
+Progress: time: Thu, 10 Oct 2013 19:30:59 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:31:29 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:31:59 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:32:29 +0000 Finished successfully:16
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-1932-uiae1df2
+Progress: time: Thu, 10 Oct 2013 19:32:57 +0000
+Progress: time: Thu, 10 Oct 2013 19:33:09 +0000 Stage in:1 Submitted:9
+Number of nodes on beagle is 0
+Nothing to do on beagle
+Number of nodes on midway is 4
+Node info for midway : midway midway011.rcc.local /dev/shm/input_25712.txt /dev/shm/input_25069.txt
+Node info for midway : midway midway009.rcc.local /dev/shm/input_29182.txt /dev/shm/input_30328.txt /dev/shm/input_16117.txt
+Node info for midway : midway midway012.rcc.local /dev/shm/input_14457.txt /dev/shm/input_14422.txt
+Node info for midway : midway midway010.rcc.local /dev/shm/input_7306.txt /dev/shm/input_26900.txt /dev/shm/input_31621.txt
+Progress: time: Thu, 10 Oct 2013 19:33:11 +0000 Active:3 Stage out:1 Finished successfully:12
+Progress: time: Thu, 10 Oct 2013 19:33:27 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:33:57 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:34:27 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:34:57 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:35:27 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:35:57 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:36:27 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:36:57 +0000 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 19:37:27 +0000 Finished successfully:16
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter uniq, on line 19, shadows variable of same name on line 122
+Warning: Parameter deref, on line 34, shadows variable of same name on line 125
+Warning: Parameter deref, on line 49, shadows variable of same name on line 125
+Warning: Parameter reducer, on line 54, shadows variable of same name on line 124
+Warning: Parameter reducer, on line 86, shadows variable of same name on line 124
+Could not start execution
+ Compile error in foreach statement at line 145
+ Compile error in procedure invocation at line 175
+ Variable n was not declared in this scope.
+real 1.19
+user 2.18
+sys 0.12
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2059-7beus7a9
+Progress: time: Thu, 10 Oct 2013 20:59:59 +0000
+Progress: time: Thu, 10 Oct 2013 21:00:06 +0000 Stage in:1 Submitted:9
+Execution failed:
+ Exception in cat:
+ Arguments: [map0000.out map0001.out map0002.out map0003.out map0004.out map0005.out map0006.out map0007.out map0008.out map0009.out]
+ Host: local
+ Directory: swiftreduce-20131010-2059-7beus7a9/jobs/z/cat-ztvl5hgl
+
+Caused by:
+ No 'proxy' provider or alias found. Available providers: [cobalt, ssh-cl, gsiftp, coaster, webdav, slurm, dcache, ssh, gt2, condor, http, coaster-persistent, pbs, ftp, lsf, gsiftp-old, local, sge]. Aliases: condor <-> condorlocal; cobalt <-> cobaltlocal; gsiftp <-> gridftp; pbs <-> pbslocal; gsiftp-old <-> gridftp-old; local <-> file;
+ aggregate, swiftreduce.swift, line 41
+real 10.38
+user 16.74
+sys 3.17
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2101-yypcdni0
+Progress: time: Thu, 10 Oct 2013 21:01:28 +0000
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2101-9s9yp0sa
+Progress: time: Thu, 10 Oct 2013 21:01:42 +0000
+Progress: time: Thu, 10 Oct 2013 21:01:59 +0000 Stage in:1 Submitted:9
+Execution failed:
+ Exception in cat:
+ Arguments: [map0000.out map0001.out map0002.out map0003.out map0004.out map0005.out map0006.out map0007.out map0008.out map0009.out]
+ Host: local
+ Directory: swiftreduce-20131010-2101-9s9yp0sa/jobs/i/cat-inao5hgl
+
+Caused by:
+ No 'proxy' provider or alias found. Available providers: [cobalt, ssh-cl, gsiftp, coaster, webdav, slurm, dcache, ssh, gt2, condor, http, coaster-persistent, pbs, ftp, lsf, gsiftp-old, local, sge]. Aliases: condor <-> condorlocal; cobalt <-> cobaltlocal; gsiftp <-> gridftp; pbs <-> pbslocal; gsiftp-old <-> gridftp-old; local <-> file;
+ aggregate, swiftreduce.swift, line 41
+real 19.71
+user 34.80
+sys 7.47
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2104-qip5f2ke
+Progress: time: Thu, 10 Oct 2013 21:04:56 +0000
+Progress: time: Thu, 10 Oct 2013 21:04:59 +0000 Stage in:1 Submitted:9
+Execution failed:
+ Exception in cat:
+ Arguments: [map0000.out map0001.out map0002.out map0003.out map0004.out map0005.out map0006.out map0007.out map0008.out map0009.out]
+ Host: local
+ Directory: swiftreduce-20131010-2104-qip5f2ke/jobs/m/cat-m8ts5hgl
+
+Caused by:
+ Application /bin/cat failed with an exit code of 1
+ aggregate, swiftreduce.swift, line 41
+real 6.15
+user 10.87
+sys 1.64
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2114-slsf4a0d
+Progress: time: Thu, 10 Oct 2013 21:14:02 +0000
+Execution failed:
+ Exception in cat:
+ Arguments: []
+ Host: local
+ Directory: swiftreduce-20131010-2114-slsf4a0d/jobs/c/cat-c4k66hgl
+
+Caused by:
+ Application /bin/cat failed with an exit code of 1
+ aggregate, swiftreduce.swift, line 43
+real 3.69
+user 4.03
+sys 0.20
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2114-iiiraxq8
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Thu, 10 Oct 2013 21:15:00 +0000
+
+Execution failed:
+Exception in cat:
+ Arguments: []
+ Host: local
+ Directory: swiftreduce-20131010-2114-iiiraxq8/jobs/3/cat-3vx76hgl
+ exception @ swift-int-staging.k, line: 162
+Caused by: Application /bin/cat failed with an exit code of 1
+ exception @ swift-int-staging.k, line: 158
+
+real 3.46
+user 2.92
+sys 0.18
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2116-y1x5n6d8
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Thu, 10 Oct 2013 21:16:21 +0000
+
+Execution failed:
+Exception in cat:
+ Arguments: []
+ Host: local
+ Directory: swiftreduce-20131010-2116-y1x5n6d8/jobs/n/cat-n1t96hgl
+ exception @ swift-int-staging.k, line: 162
+Caused by: Application /bin/cat failed with an exit code of 1
+ exception @ swift-int-staging.k, line: 158
+
+real 2.34
+user 2.91
+sys 0.17
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:10
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2117-hng5n22a
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Thu, 10 Oct 2013 21:17:06 +0000
+Progress: time: Thu, 10 Oct 2013 21:17:07 +0000 Submitted:20
+
+Execution failed:
+Exception in cat:
+ Arguments: [map0000.out map0001.out map0002.out map0003.out map0004.out map0005.out map0006.out map0007.out map0008.out map0009.out map0010.out map0011.out map0012.out map0013.out map0014.out map0015.out map0016.out map0017.out map0018.out map0019.out]
+ Host: local
+ Directory: swiftreduce-20131010-2117-hng5n22a/jobs/o/cat-ooua6hgl
+ exception @ swift-int-staging.k, line: 162
+Caused by: Application /bin/cat failed with an exit code of 1
+ exception @ swift-int-staging.k, line: 158
+
+Progress: time: Thu, 10 Oct 2013 21:17:12 +0000 Failed:1 Finished successfully:20
+real 8.20
+user 3.94
+sys 0.24
+chunks 20 filesize 10 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:3 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2118-97ej2zb6
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Thu, 10 Oct 2013 21:18:09 +0000
+Progress: time: Thu, 10 Oct 2013 21:18:10 +0000 Submitted:3
+Progress: time: Thu, 10 Oct 2013 21:18:14 +0000 Submitted:1 Finished successfully:3
+
+Execution failed:
+Exception in cat:
+ Arguments: [map0000.out map0001.out map0002.out]
+ Host: local
+ Directory: swiftreduce-20131010-2118-97ej2zb6/jobs/e/cat-embc6hgl
+ exception @ swift-int-staging.k, line: 162
+Caused by: Application /bin/cat failed with an exit code of 1
+ exception @ swift-int-staging.k, line: 158
+
+real 7.21
+user 3.22
+sys 0.21
+chunks 3 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 30000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:3 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2120-z0ndz0t9
+Warning: The @ syntax for function invocation is deprecated
+[Error] multiple.xml, line 19, col 38: cvc-complex-type.4: Attribute 'major' must appear on element 'gridftp'.
+[Error] multiple.xml, line 19, col 38: cvc-complex-type.4: Attribute 'minor' must appear on element 'gridftp'.
+Progress: time: Thu, 10 Oct 2013 21:20:54 +0000
+Progress: time: Thu, 10 Oct 2013 21:20:55 +0000 Submitted:3
+
+Execution failed:
+Exception in cat:
+ Arguments: [map0000.out map0001.out map0002.out]
+ Host: local
+ Directory: swiftreduce-20131010-2120-z0ndz0t9/jobs/3/cat-3k6g6hgl
+ exception @ swift-int-staging.k, line: 162
+Caused by: Application /bin/cat failed with an exit code of 1
+ exception @ swift-int-staging.k, line: 158
+
+Progress: time: Thu, 10 Oct 2013 21:21:07 +0000 Failed:1 Finished successfully:3
+real 15.09
+user 3.25
+sys 0.20
+chunks 3 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 30000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:3 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2122-bg7ffgcd
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Thu, 10 Oct 2013 21:22:34 +0000
+Progress: time: Thu, 10 Oct 2013 21:22:35 +0000 Submitted:3
+Progress: time: Thu, 10 Oct 2013 21:22:40 +0000 Stage in:1 Submitted:2
+
+Execution failed:
+Exception in cat:
+ Arguments: [map0000.out map0001.out map0002.out]
+ Host: local
+ Directory: swiftreduce-20131010-2122-bg7ffgcd/jobs/m/cat-m9ii6hgl
+ exception @ swift-int-staging.k, line: 162
+Caused by: Application /bin/cat failed with an exit code of 1
+ exception @ swift-int-staging.k, line: 158
+
+real 8.33
+user 3.25
+sys 0.19
+chunks 3 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 30000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2124-ada37hs3
+Warning: The @ syntax for function invocation is deprecated
+Could not start execution:
+ Compile error in procedure invocation at line 42:
+ Wrong number of procedure input arguments: specified 1 and should be 2
+real 1.72
+user 1.58
+sys 0.09
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:beagle, CHUNKS:20 X FILESIZE:10
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Could not start execution
+ Compile error in foreach statement at line 53
+ Compile error in procedure invocation at line 54
+ Procedure reduce is not declared.
+real 1.60
+user 1.54
+sys 0.11
+chunks 20 filesize 10 site beagle type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:beagle, CHUNKS:20 X FILESIZE:10
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2141-dnt1bqb8
+Progress: time: Thu, 10 Oct 2013 21:41:58 +0000
+Execution failed:
+ The application "bash" is not available for any site/pool in your tc.data catalog
+ map_data, swiftreduce.swift, line 38
+real 2.33
+user 4.15
+sys 0.15
+chunks 20 filesize 10 site beagle type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 2000000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2142-9oovj0g0
+Progress: time: Thu, 10 Oct 2013 21:42:09 +0000
+Progress: time: Thu, 10 Oct 2013 21:42:17 +0000 Stage in:1 Submitted:9
+Final status: Thu, 10 Oct 2013 21:42:17 +0000 Finished successfully:11
+real 10.88
+user 19.76
+sys 3.30
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:20 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2144-z7q0vegc
+Progress: time: Thu, 10 Oct 2013 21:44:43 +0000
+Progress: time: Thu, 10 Oct 2013 21:44:56 +0000 Submitting:19 Submitted:1
+Progress: time: Thu, 10 Oct 2013 21:45:04 +0000 Stage in:1 Submitted:19
+Progress: time: Thu, 10 Oct 2013 21:45:13 +0000 Submitted:4 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 21:45:18 +0000 Stage in:1 Submitted:3 Finished successfully:16
+Progress: time: Thu, 10 Oct 2013 21:45:20 +0000 Active:1 Finished successfully:20
+Final status: Thu, 10 Oct 2013 21:45:20 +0000 Finished successfully:21
+real 39.51
+user 67.20
+sys 16.31
+chunks 20 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 200000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2303-0zj61zlc
+Progress: time: Thu, 10 Oct 2013 23:03:31 +0000
+Progress: time: Thu, 10 Oct 2013 23:03:47 +0000 Submitting:6 Submitted:4
+Progress: time: Thu, 10 Oct 2013 23:03:54 +0000 Stage in:1 Submitted:9
+Progress: time: Thu, 10 Oct 2013 23:04:01 +0000 Submitted:4 Finished successfully:6
+Progress: time: Thu, 10 Oct 2013 23:04:31 +0000 Submitted:4 Finished successfully:6
+Progress: time: Thu, 10 Oct 2013 23:05:01 +0000 Submitted:4 Finished successfully:6
+Progress: time: Thu, 10 Oct 2013 23:05:31 +0000 Submitted:4 Finished successfully:6
+Progress: time: Thu, 10 Oct 2013 23:06:01 +0000 Submitted:4 Finished successfully:6
+Progress: time: Thu, 10 Oct 2013 23:06:31 +0000 Submitted:4 Finished successfully:6
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2312-s087ojf4
+Progress: time: Thu, 10 Oct 2013 23:13:01 +0000
+Progress: time: Thu, 10 Oct 2013 23:13:13 +0000 Submitting:5 Submitted:5
+Progress: time: Thu, 10 Oct 2013 23:13:21 +0000 Stage in:1 Submitted:9
+Progress: time: Thu, 10 Oct 2013 23:13:31 +0000 Submitted:4 Finished successfully:6
+Progress: time: Thu, 10 Oct 2013 23:14:01 +0000 Submitted:4 Finished successfully:6
+Progress: time: Thu, 10 Oct 2013 23:14:31 +0000 Submitted:4 Finished successfully:6
+Progress: time: Thu, 10 Oct 2013 23:15:01 +0000 Submitted:4 Finished successfully:6
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131010-2316-x5ida1da
+Progress: time: Thu, 10 Oct 2013 23:16:04 +0000
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2317-2adst5fd
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Thu, 10 Oct 2013 23:17:13 +0000
+Progress: time: Thu, 10 Oct 2013 23:17:14 +0000 Submitting:10
+Progress: time: Thu, 10 Oct 2013 23:17:28 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 23:17:35 +0000 Stage in:1 Finished successfully:10
+Progress: time: Thu, 10 Oct 2013 23:17:36 +0000 Stage in:2 Finished successfully:11
+Final status: Thu, 10 Oct 2013 23:17:37 +0000 Finished successfully:13
+real 26.11
+user 5.30
+sys 0.29
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2323-x8kmsdnd
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Thu, 10 Oct 2013 23:23:35 +0000
+Progress: time: Thu, 10 Oct 2013 23:23:36 +0000 Submitting:10
+Progress: time: Thu, 10 Oct 2013 23:23:47 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 23:23:55 +0000 Stage in:1 Finished successfully:10
+Progress: time: Thu, 10 Oct 2013 23:23:56 +0000 Submitting:2 Finished successfully:11
+Final status: Thu, 10 Oct 2013 23:23:56 +0000 Finished successfully:13
+real 23.62
+user 5.10
+sys 0.28
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131010-2336-zn2cbxs0
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Thu, 10 Oct 2013 23:36:03 +0000
+Progress: time: Thu, 10 Oct 2013 23:36:04 +0000 Submitting:10
+Progress: time: Thu, 10 Oct 2013 23:36:16 +0000 Submitted:10
+Progress: time: Thu, 10 Oct 2013 23:36:23 +0000 Active:1 Stage out:9
+Progress: time: Thu, 10 Oct 2013 23:36:24 +0000 Stage in:1 Finished successfully:10
+Final status: Thu, 10 Oct 2013 23:36:25 +0000 Finished successfully:13
+real 24.01
+user 5.18
+sys 0.28
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Warning: Parameter reducer, on line 50, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 50, shadows variable of same name on line 43
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131011-0114-pldfvol9
+Progress: time: Fri, 11 Oct 2013 01:14:46 +0000
+Progress: time: Fri, 11 Oct 2013 01:14:58 +0000 Submitting:9 Submitted:1
+Progress: time: Fri, 11 Oct 2013 01:15:05 +0000 Stage in:1 Submitted:9
+Progress: time: Fri, 11 Oct 2013 01:15:07 +0000 Active:1 Finished successfully:10
+Progress: time: Fri, 11 Oct 2013 01:15:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:15:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:16:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:16:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:17:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:17:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:18:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:18:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:19:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:19:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:20:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:20:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:21:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:21:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:22:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:22:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:23:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:23:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:24:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:24:46 +0000 Submitted:1 Finished successfully:12
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Warning: Parameter reducer, on line 50, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 50, shadows variable of same name on line 43
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131011-0125-2il2vmvb
+Progress: time: Fri, 11 Oct 2013 01:25:06 +0000
+Progress: time: Fri, 11 Oct 2013 01:25:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:25:18 +0000 Submitting:7 Submitted:1 Finished successfully:2
+Progress: time: Fri, 11 Oct 2013 01:25:28 +0000 Stage in:1 Submitted:6 Finished successfully:3
+Progress: time: Fri, 11 Oct 2013 01:25:29 +0000 Active:1 Finished successfully:10
+Final status: Fri, 11 Oct 2013 01:25:30 +0000 Finished successfully:13
+real 26.51
+user 45.42
+sys 10.62
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+Progress: time: Fri, 11 Oct 2013 01:25:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:26:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:26:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:27:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:27:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:28:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:28:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:29:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:29:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:30:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:30:46 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:31:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:31:46 +0000 Submitted:1 Finished successfully:12
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Warning: Parameter reducer, on line 50, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 50, shadows variable of same name on line 43
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131011-0131-37x4zvy5
+Progress: time: Fri, 11 Oct 2013 01:32:01 +0000
+Progress: time: Fri, 11 Oct 2013 01:32:13 +0000 Submitting:4 Submitted:1 Finished successfully:5
+Progress: time: Fri, 11 Oct 2013 01:32:16 +0000 Submitted:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:32:20 +0000 Stage in:1 Submitted:4 Finished successfully:5
+Final status: Fri, 11 Oct 2013 01:32:21 +0000 Finished successfully:13
+real 22.29
+user 39.59
+sys 9.53
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+Progress: time: Fri, 11 Oct 2013 01:32:46 +0000 Submitted:1 Finished successfully:12
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Warning: Parameter reducer, on line 50, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 50, shadows variable of same name on line 43
+Warning: Parameter reducer, on line 53, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 53, shadows variable of same name on line 43
+Could not start execution
+ Compile error in procedure invocation at line 79
+ Wrong type for parameter number 2, expected fileptr, got file[int]
+real 1.05
+user 1.68
+sys 0.11
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Warning: Parameter reducer, on line 50, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 50, shadows variable of same name on line 43
+Warning: Parameter reducer, on line 53, shadows variable of same name on line 30
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131011-0141-ykjofv7b
+Progress: time: Fri, 11 Oct 2013 01:41:58 +0000
+Progress: time: Fri, 11 Oct 2013 01:42:10 +0000 Submitting:9 Submitted:1
+Progress: time: Fri, 11 Oct 2013 01:42:17 +0000 Stage in:1 Submitted:5 Finished successfully:4
+Progress: time: Fri, 11 Oct 2013 01:42:19 +0000 Active:1 Finished successfully:10
+Progress: time: Fri, 11 Oct 2013 01:42:28 +0000 Finished successfully:13
+Progress: time: Fri, 11 Oct 2013 01:42:58 +0000 Finished successfully:13
+Progress: time: Fri, 11 Oct 2013 01:43:28 +0000 Finished successfully:13
+Progress: time: Fri, 11 Oct 2013 01:43:58 +0000 Finished successfully:13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Warning: Parameter aggr, on line 17, shadows variable of same name on line 32
+Warning: Parameter reducer, on line 47, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 47, shadows variable of same name on line 43
+Warning: Parameter reducer, on line 50, shadows variable of same name on line 30
+Warning: Parameter all_map, on line 50, shadows variable of same name on line 43
+Warning: Parameter reducer, on line 53, shadows variable of same name on line 30
+Swift 0.94.1 RC4 swift-r7109 cog-r3802
+
+RunID: 20131011-0144-hbtdsex2
+Progress: time: Fri, 11 Oct 2013 01:44:16 +0000
+Progress: time: Fri, 11 Oct 2013 01:44:29 +0000 Submitting:7 Submitted:1 Finished successfully:2
+Progress: time: Fri, 11 Oct 2013 01:44:35 +0000 Stage in:1 Submitted:5 Finished successfully:4
+Progress: time: Fri, 11 Oct 2013 01:44:46 +0000 Finished successfully:13
+Progress: time: Fri, 11 Oct 2013 01:45:16 +0000 Finished successfully:13
+Progress: time: Fri, 11 Oct 2013 01:45:46 +0000 Finished successfully:13
+Progress: time: Fri, 11 Oct 2013 01:46:16 +0000 Finished successfully:13
+Progress: time: Fri, 11 Oct 2013 01:46:46 +0000 Finished successfully:13
+Progress: time: Fri, 11 Oct 2013 01:47:16 +0000 Finished successfully:13
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131011-0147-yc0mur7g
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Fri, 11 Oct 2013 01:47:52 +0000
+Progress: time: Fri, 11 Oct 2013 01:47:53 +0000 Submitting:6 Finished successfully:4
+Progress: time: Fri, 11 Oct 2013 01:48:05 +0000 Submitted:5 Finished successfully:5
+Progress: time: Fri, 11 Oct 2013 01:48:13 +0000 Stage in:1 Finished successfully:10
+Progress: time: Fri, 11 Oct 2013 01:48:14 +0000 Stage in:1 Finished successfully:12
+Progress: time: Fri, 11 Oct 2013 01:48:15 +0000 Finished successfully:13
+No events in 1s.
+Finding dependency loops...
+
+Waiting threads:
+Thread: R-15, waiting on redsite_out (declared on line 57)
+ swift:execute, swiftreduce, line 216
+ reduce_loc, swiftreduce, line 408
+
+----
+No dependency loops found.
+
+The following threads are independently hung:
+Thread: R-15, waiting on redsite_out (declared on line 57)
+ swift:execute, swiftreduce, line 216
+ reduce_loc, swiftreduce, line 408
+
+----
+
+Irrecoverable error found. Exiting.
+real 26.05
+user 5.44
+sys 0.33
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT does *NOT* match expected records
+ACTUAL COUNT :
+EXPECTED COUNT: 100000
+---------------------------------------------
+================================================================
+RUNTYPE :SWIFT+Chirp (With local combiners), SITE:multiple, CHUNKS:10 X FILESIZE:1
+Swift trunk swift-r7138 cog-r3806
+RunID: 20131011-0150-f46n851d
+Warning: The @ syntax for function invocation is deprecated
+Progress: time: Fri, 11 Oct 2013 01:50:03 +0000
+Progress: time: Fri, 11 Oct 2013 01:50:04 +0000 Submitting:4 Submitted:6
+Progress: time: Fri, 11 Oct 2013 01:50:07 +0000 Submitting:4 Finished successfully:6
+Progress: time: Fri, 11 Oct 2013 01:50:16 +0000 Submitted:4 Finished successfully:6
+Progress: time: Fri, 11 Oct 2013 01:50:23 +0000 Submitting:1 Submitted:1 Finished successfully:11
+Final status: Fri, 11 Oct 2013 01:50:23 +0000 Finished successfully:14
+real 22.86
+user 14.06
+sys 5.86
+chunks 10 filesize 1 site multiple type Swift+Chirp
+---------------------------------------------
+COUNT matches expected records
+ACTUAL COUNT : 100000
+EXPECTED COUNT: 100000
+---------------------------------------------
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/README
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/README (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/README 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,47 @@
+Swift-Reduce
+============
+
+Run MapReduce style applications with Swift. Since swift is a file based
+model as opposed to the Key-Value record based approach of MapReduce, there
+are differences in the way the user supplied Map and Reduce functions
+should work.
+
+map_core.sh
+-----------
+
+The user supplied Map function should be a bash function named mapper().
+It would take any number of arguments from swift and would output it's
+results to stdout. Swift would redirect the output to appropriate files.
+The mapper function should be placed in a file map_core.sh.
+
+reduce_core.sh
+--------------
+
+The Reduce function should be named reduce() and placed in reduce_core.sh.
+This function would be passed intermediate files generated from the map
+stage and output its results to a file of its choosing. The filename of
+the result file is output to stdout.
+
+setup.sh
+--------
+
+Run the setup.sh script to embed the map_core.sh and reduce_core.sh scripts
+in the wrapper scripts used by swift in runtime.
+
+
+Example - Teragen
+=================
+
+Record
+======
+Each record consists of 1 random number and count, padded to 100 bytes.
+So, in the usual run a file of 10^8 records generated would have 10^6 bytes.
+
+
+Describe workflow
+=================
+
+teragen will split the task of generating files containing 10^8 records (~100 MB)
+over 10000 tasks, thus generating in total 1 Tera records.
+
+./teragen_wrap.sh 10 # Generates 10000000
\ No newline at end of file
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/aggregate.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/aggregate.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/aggregate.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,3 @@
+#!/bin/bash
+
+cat $*
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/aggregate.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/beagle.xml
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/beagle.xml (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/beagle.xml 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,24 @@
+<config>
+ <pool handle="beagle">
+ <execution provider="coaster" jobmanager="local:pbs"/>
+ <profile namespace="globus" key="jobsPerNode">1</profile>
+ <profile namespace="globus" key="lowOverAllocation">100</profile>
+ <profile namespace="globus" key="highOverAllocation">100</profile>
+ <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
+ <!-- to use a beage reservation, eg:
+ <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=24;pbs.resource_list=advres=wilde.1768</profile>
+ -->
+ <profile namespace="globus" key="maxtime">2400</profile>
+ <profile namespace="globus" key="maxWalltime">00:20:00</profile>
+ <profile namespace="globus" key="queue">batch</profile>
+ <profile namespace="globus" key="userHomeOverride">/lustre/beagle/yadunandb/swiftwork</profile>
+ <profile namespace="globus" key="slots">10</profile>
+ <profile namespace="globus" key="maxnodes">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="karajan" key="jobThrottle">1.00</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <!-- <filesystem provider="local"/> -->
+ <!-- <workdirectory>/lustre/beagle/yadunandb/swiftwork</workdirectory> -->
+ <workdirectory>/tmp/yadunandb/swiftwork</workdirectory>
+ </pool>
+</config>
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/clean.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/clean.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/clean.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,9 @@
+#!/bin/bash
+
+rm -rf *log *d &> /dev/null
+rm -rf *swiftx *kml &> /dev/null
+rm -rf *~ &> /dev/null
+rm *{out,err,imd2} &> /dev/null
+rm final_errors final_result &> /dev/null
+rm exec_client exec_server &> /dev/null
+rm -rf swiftreduce-*-* &> /dev/null
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/clean.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/client.c
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/client.c (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/client.c 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,131 @@
+/*
+** client.c -- a stream socket client demo
+* Thanks to Brian "Beej Jorgensen" Hall
+* Hacked to work as a remote exec server by Yadu
+* <yadunandb at ci.uchicago.edu>
+*/
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <netdb.h>
+#include <sys/types.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+#include <string.h>
+
+#include <arpa/inet.h>
+
+#define PORT "3490" // the port client will be connecting to
+
+#define MAXDATASIZE 2048 // max number of bytes we can get at once
+
+// get sockaddr, IPv4 or IPv6:
+void *get_in_addr(struct sockaddr *sa)
+{
+ if (sa->sa_family == AF_INET) {
+ return &(((struct sockaddr_in*)sa)->sin_addr);
+ }
+ return &(((struct sockaddr_in6*)sa)->sin6_addr);
+}
+
+
+int main(int argc, char *argv[])
+{
+ int sockfd, new_fd, numbytes;
+ int filelen;
+ struct addrinfo hints, *servinfo, *p;
+ int rv;
+ char s[INET6_ADDRSTRLEN];
+ FILE *f_in;
+ int file_size;
+ char *buffer;
+ char *reply_buf;
+ int con_res;
+ if (argc != 3) {
+ fprintf(stderr,"usage: client hostname remote_script\n");
+ exit(1);
+ }
+
+ memset(&hints, 0, sizeof hints);
+ hints.ai_family = AF_UNSPEC;
+ hints.ai_socktype = SOCK_STREAM;
+
+
+ // Leaving PORT as NULL
+ if ((rv = getaddrinfo(argv[1], PORT, &hints, &servinfo)) != 0) {
+ fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rv));
+ return 1;
+ }
+
+ // loop through all the results and connect to the first we can
+ for(p = servinfo; p != NULL; p = p->ai_next) {
+ if ((sockfd = socket(p->ai_family, p->ai_socktype,
+ p->ai_protocol)) == -1) {
+ perror("client: socket");
+ continue;
+ }
+
+ if ( (con_res = connect(sockfd, p->ai_addr, p->ai_addrlen)) == -1) {
+ close(sockfd);
+ fprintf(stderr,"Connect failed with %d:%s\n", con_res, strerror(errno));
+ perror("client: connect");
+ continue;
+ }
+
+ break;
+ }
+
+ if (p == NULL) {
+ fprintf(stderr, "client: failed to connect\n");
+ return 2;
+ }
+
+ inet_ntop(p->ai_family, get_in_addr((struct sockaddr *)p->ai_addr),
+ s, sizeof s);
+ fprintf(stderr,"client: connecting to %s\n", s);
+
+ freeaddrinfo(servinfo); // all done with this structure
+
+ //Send message to server
+ fprintf(stderr,"Client sending contents of file %s to server \n",argv[2]);
+
+ // Read file contents into a buffer;
+ if ( (f_in = fopen(argv[2],"r")) == NULL ){
+ fprintf(stderr, "Opening[%s] failed: %s\n", argv[2], strerror(errno));
+ exit(errno);
+ };
+
+ fseek(f_in, 0, SEEK_END);
+ file_size = ftell(f_in); // Find length
+ fprintf(stderr, "File size: %d\n", file_size);
+ fseek(f_in, 0, SEEK_SET);
+
+ buffer = (char *) malloc ((sizeof (char)) * (file_size+2));
+ fread(buffer, file_size, 1, f_in);
+ buffer[file_size] = '\0';
+ fclose(f_in);
+
+ fprintf(stderr, "Content of file : \n%s\n", buffer);
+
+ if (send(sockfd, buffer, file_size, 0) == -1)
+ perror("send");
+ free(buffer);
+
+ //Receive message back from server
+ reply_buf = (char *)malloc(MAXDATASIZE);
+ fprintf(stderr, "Client receiving message from server \n");
+ if ((numbytes = recv(sockfd, reply_buf, MAXDATASIZE-1, 0)) == -1) {
+ perror("recv");
+ exit(1);
+ }
+ //printf("Numbytes recv'd : %d\n", numbytes);
+ reply_buf[numbytes] = '\0';
+ printf("%s",reply_buf);
+ free(reply_buf);
+ close(sockfd);
+
+ return 0;
+}
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,40 @@
+#!/bin/bash
+
+ARGS=($*)
+SUM=0
+COUNT=0
+
+# When strings in format node files... are passed to combiner
+# 1. Run a local_combiner on the remote node
+# 2. Return a file pointer
+NODE=${ARGS[0]}
+FILES=(${ARGS[*]:1})
+REMOTE_PORT=29900
+HOSTNAME=$(hostname -f)
+
+############## LOCAL COMBINER CODE ###################
+cat << EOF > $NODE.sh
+#!/bin/bash
+ID=$RANDOM
+FILES=(${FILES[*]})
+EOF
+cat <<'EOF' >> $NODE.sh
+#USER_REDUCE
+reduce(){
+ FILES=($*)
+ DIRNAME=$(dirname ${FILES[0]})
+ RESULT=$DIRNAME/$(hostname -f).$RANDOM.imd
+ awk '{ sum += $1; count += $2 } END { print sum,count }' ${FILES[*]} > $RESULT
+ echo $RESULT
+ rm -rf ${FILES[*]} &> /dev/null
+}
+echo -e "$(hostname -f) $(reduce ${FILES[*]})"
+EOF
+chmod a+x $NODE.sh
+############# END LOCAL COMBINER #####################
+cp /lustre/beagle/yadunandb/bin/exec_client .
+echo "Combiner ${ARGS[*]}" 1>&2
+echo "Combiner [./exec_client $NODE $NODE.sh]" 1>&2
+cat $NODE.sh 1>&2
+./exec_client $NODE $NODE.sh
+exit 0
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner_template.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner_template.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner_template.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+ARGS=($*)
+SUM=0
+COUNT=0
+
+# When strings in format node files... are passed to combiner
+# 1. Run a local_combiner on the remote node
+# 2. Return a file pointer
+NODE=${ARGS[0]}
+FILES=(${ARGS[*]:1})
+REMOTE_PORT=29900
+HOSTNAME=$(hostname -f)
+
+############## LOCAL COMBINER CODE ###################
+cat << EOF > $NODE.sh
+#!/bin/bash
+ID=$RANDOM
+FILES=(${FILES[*]})
+EOF
+cat <<'EOF' >> $NODE.sh
+#USER_REDUCE
+echo -e "$(hostname -f) $(reduce ${FILES[*]})"
+EOF
+chmod a+x $NODE.sh
+############# END LOCAL COMBINER #####################
+cp /lustre/beagle/yadunandb/bin/exec_client .
+echo "Combiner ${ARGS[*]}" 1>&2
+echo "Combiner [./exec_client $NODE $NODE.sh]" 1>&2
+cat $NODE.sh 1>&2
+./exec_client $NODE $NODE.sh
+exit 0
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/combiner_template.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/compile
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/compile (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/compile 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,18 @@
+#!/bin/bash
+
+gcc server.c -o exec_server
+gcc client.c -o exec_client
+
+chmod a+x exec_server
+chmod a+x exec_client
+
+if [[ "$HOSTNAME" == *midway* ]]; then
+ cp exec_server /scratch/midway/yadunand/bin/
+ cp exec_client /scratch/midway/yadunand/bin/
+ echo "Copied to /scratch/midway/yadunandb/bin/ on Midway"
+
+elif [[ "$HOSTNAME" == *beagle* ]]; then
+ cp exec_server /lustre/beagle/yadunandb/bin/
+ cp exec_client /lustre/beagle/yadunandb/bin/
+ echo "Copied to /lustre/beagle/yadunandb/bin/ on Beagle"
+fi
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/compile
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/dereference.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/dereference.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/dereference.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,20 @@
+#!/bin/bash
+# The derefernce script takes one fileptr as input
+# and dumps the content of the file to stdout.
+TOKEN=($(cat $1))
+REMOTE=${TOKEN[0]}
+FILE=${TOKEN[1]}
+TARGETFILE=$(basename $FILE)
+REPODIR=$(dirname $FILE)
+echo "$REMOTE $HOSTNAME" 1>&2
+if [ "$REMOTE" == "$(hostname -f)" ]
+then # Move to an identifiable file if local
+ cat $FILE
+else # Fetch to an identifiable file if remote
+ echo "chirp_get $REMOTE $TARGETFILE $FILE" 1>&2
+ chirp_get $REMOTE $TARGETFILE $FILE
+ cat $FILE
+ rm $FILE
+fi
+
+
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/dereference.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/map_core.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/map_core.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/map_core.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,13 @@
+mapper (){
+ ARG1=$1;
+ LOWERLIMIT=0
+ UPPERLIMIT=1000000
+ SLICESIZE=10000 # 10^4 records padded to 100B would result in 1MB file
+ #Generate $SLICESIZE number of 100B records of format: RANDOM_NUMBER 1
+ shuf -i $LOWERLIMIT-$UPPERLIMIT -n $SLICESIZE | awk '{printf "%-97s %s\n", $0, 1}' > tmp_shuf
+ for i in $(seq 1 1 $ARG1)
+ do
+ cat tmp_shuf
+ done
+ rm tmp_shuf
+}
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/map_core.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,63 @@
+#!/bin/bash
+
+# By default with ARG1:100 and SLICESIZE=10000, this script will generate
+# 10^6 records.
+ARG1=1
+[ ! -z $1 ] && ARG1=$1
+
+FILE="input_$RANDOM.txt"
+HOSTNAME=$(hostname -f)
+if [[ "$HOSTNAME" == *midway* ]]; then # On midway node
+ EXECSERVER=/scratch/midway/yadunand/bin/exec_server
+ LOGFOLDER=/scratch/midway/yadunand
+ FOLDER="/dev/shm"
+ HOSTNAME="midway_$HOSTNAME"
+
+elif [[ "$HOSTNAME" == *nid* ]]; then # On beagle node
+ EXECSERVER=/lustre/beagle/yadunandb/bin/exec_server
+ LOGFOLDER=/lustre/beagle/yadunandb/Swift-MapRed/mapred_combiner_multisite
+ FOLDER="/dev/shm"
+ HOSTNAME="beagle_$HOSTNAME"
+fi
+
+TIMEOUT=1200
+NC_LISTEN_PORT=29900
+
+# Start services on Node
+JOBS_LEVEL=`echo $PWD | grep -o ".*jobs"`
+if [ "$?" == "0" ]
+then
+ mkdir $JOBS_LEVEL/CHIRPING
+ if [ "$?" == "0" ]
+ then
+ killall -u $USER chirp_server;
+ echo "unix:$USER rwlds" > $FOLDER/acl.conf
+ echo "hostname:* rwl" >> $FOLDER/acl.conf
+ which chirp_server 1>&2
+ timeout $TIMEOUT chirp_server -A $FOLDER/acl.conf -r $FOLDER &
+ timeout $TIMEOUT $EXECSERVER &> $LOGFOLDER/exec_server_log-$HOSTNAME.log &
+ fi
+ FILE=$FOLDER/$FILE;
+else
+ FILE=$PWD/$FILE
+fi
+
+#USER_MAP
+mapper (){
+ ARG1=$1;
+ LOWERLIMIT=0
+ UPPERLIMIT=1000000
+ SLICESIZE=10000 # 10^4 records padded to 100B would result in 1MB file
+ #Generate $SLICESIZE number of 100B records of format: RANDOM_NUMBER 1
+ shuf -i $LOWERLIMIT-$UPPERLIMIT -n $SLICESIZE | awk '{printf "%-97s %s\n", $0, 1}' > tmp_shuf
+ for i in $(seq 1 1 $ARG1)
+ do
+ cat tmp_shuf
+ done
+ rm tmp_shuf
+}
+
+mapper $ARG1 > $FILE
+echo "$HOSTNAME $FILE"
+ps -u $USER 1>&2
+exit 0;
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper_template.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper_template.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper_template.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,50 @@
+#!/bin/bash
+
+# By default with ARG1:100 and SLICESIZE=10000, this script will generate
+# 10^6 records.
+ARG1=1
+[ ! -z $1 ] && ARG1=$1
+
+FILE="input_$RANDOM.txt"
+HOSTNAME=$(hostname -f)
+if [[ "$HOSTNAME" == *midway* ]]; then # On midway node
+ EXECSERVER=/scratch/midway/yadunand/bin/exec_server
+ LOGFOLDER=/scratch/midway/yadunand
+ FOLDER="/dev/shm"
+ HOSTNAME="midway_$HOSTNAME"
+
+elif [[ "$HOSTNAME" == *nid* ]]; then # On beagle node
+ EXECSERVER=/lustre/beagle/yadunandb/bin/exec_server
+ LOGFOLDER=/lustre/beagle/yadunandb/Swift-MapRed/mapred_combiner_multisite
+ FOLDER="/dev/shm"
+ HOSTNAME="beagle_$HOSTNAME"
+fi
+
+TIMEOUT=1200
+NC_LISTEN_PORT=29900
+
+# Start services on Node
+JOBS_LEVEL=`echo $PWD | grep -o ".*jobs"`
+if [ "$?" == "0" ]
+then
+ mkdir $JOBS_LEVEL/CHIRPING
+ if [ "$?" == "0" ]
+ then
+ killall -u $USER chirp_server;
+ echo "unix:$USER rwlds" > $FOLDER/acl.conf
+ echo "hostname:* rwl" >> $FOLDER/acl.conf
+ which chirp_server 1>&2
+ timeout $TIMEOUT chirp_server -A $FOLDER/acl.conf -r $FOLDER &
+ timeout $TIMEOUT $EXECSERVER &> $LOGFOLDER/exec_server_log-$HOSTNAME.log &
+ fi
+ FILE=$FOLDER/$FILE;
+else
+ FILE=$PWD/$FILE
+fi
+
+#USER_MAP
+
+mapper $ARG1 > $FILE
+echo "$HOSTNAME $FILE"
+ps -u $USER 1>&2
+exit 0;
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/mapper_template.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/midway.xml
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/midway.xml (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/midway.xml 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,17 @@
+<config>
+ <pool handle="midway">
+ <execution provider="coaster" jobmanager="local:slurm"/>
+ <profile namespace="globus" key="queue">sandyb</profile>
+ <profile namespace="globus" key="jobsPerNode">16</profile>
+ <profile namespace="globus" key="maxWalltime">00:10:00</profile>
+ <profile namespace="globus" key="maxTime">3600</profile>
+ <profile namespace="globus" key="highOverAllocation">100</profile>
+ <profile namespace="globus" key="lowOverAllocation">100</profile>
+ <profile namespace="globus" key="slots">10</profile>
+ <profile namespace="globus" key="maxNodes">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="karajan" key="jobThrottle">1.50</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <workdirectory>/tmp/yadunand/work</workdirectory>
+ </pool>
+</config>
\ No newline at end of file
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/multiple.xml
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/multiple.xml (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/multiple.xml 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,99 @@
+<?xml version="1.0" encoding="utf-8"?>
+<config xmlns="http://www.ci.uchicago.edu/swift/SwiftSites">
+ <pool handle="local">
+ <execution provider="coaster" jobmanager="local:local"/>
+ <profile namespace="globus" key="jobsPerNode">4</profile>
+ <profile namespace="globus" key="maxWalltime">00:05:00</profile>
+ <profile namespace="globus" key="highOverAllocation">100</profile>
+ <profile namespace="globus" key="lowOverAllocation">100</profile>
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="maxNodes">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="karajan" key="jobThrottle">0.03</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <filesystem provider="local"/>
+ <workdirectory>/tmp/yadunand</workdirectory>
+ </pool>
+
+ <pool handle="beagle">
+ <execution provider="coaster" jobmanager="ssh-cl:pbs" url="login4.beagle.ci.uchicago.edu"/>
+ <profile namespace="globus" key="jobsPerNode">24</profile>
+ <profile namespace="globus" key="lowOverAllocation">100</profile>
+ <profile namespace="globus" key="highOverAllocation">100</profile>
+ <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
+ <profile namespace="globus" key="maxtime">1200</profile>
+ <profile namespace="globus" key="maxWalltime">00:10:00</profile>
+ <profile namespace="globus" key="queue">development</profile>
+ <profile namespace="globus" key="userHomeOverride">/lustre/beagle/yadunandb/swiftwork</profile>
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="maxnodes">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="karajan" key="jobThrottle">4.80</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <!-- <profile namespace="karajan" key="workerLoggingLevel">trace</profile> -->
+ <profile namespace="globus" key="workerLoggingLevel">DEBUG</profile>
+ <!-- <workdirectory>/lustre/beagle/yadunandb/swiftwork</workdirectory> -->
+ <workdirectory>/tmp/yadunandb/swiftwork</workdirectory>
+ </pool>
+
+
+ <pool handle="beagle_local">
+ <execution provider="coaster" jobmanager="local:pbs"/>
+ <profile namespace="globus" key="jobsPerNode">1</profile>
+ <profile namespace="globus" key="lowOverAllocation">100</profile>
+ <profile namespace="globus" key="highOverAllocation">100</profile>
+ <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
+ <!-- to use a beage reservation, eg:
+ <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=24;pbs.resource_list=advres=wilde.1768</profile>
+ -->
+ <profile namespace="globus" key="maxtime">2400</profile>
+ <profile namespace="globus" key="maxWalltime">00:20:00</profile>
+ <profile namespace="globus" key="queue">batch</profile>
+ <profile namespace="globus" key="userHomeOverride">/lustre/beagle/yadunandb/swiftwork</profile>
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="maxnodes">4</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="karajan" key="jobThrottle">1.00</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <!-- <filesystem provider="local"/> -->
+ <!-- <workdirectory>/lustre/beagle/yadunandb/swiftwork</workdirectory> -->
+ <workdirectory>/tmp/yadunandb/swiftwork</workdirectory>
+ </pool>
+
+ <pool handle="beagle_remote">
+ <execution provider="coaster" jobmanager="ssh-cl:pbs" url="login4.beagle.ci.uchicago.edu"/>
+ <profile namespace="globus" key="jobsPerNode">4</profile>
+ <profile namespace="globus" key="lowOverAllocation">100</profile>
+ <profile namespace="globus" key="highOverAllocation">100</profile>
+ <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=24;pbs.resource_list=advres=wilde.1768</profile>
+ <profile namespace="globus" key="maxtime">1200</profile>
+ <profile namespace="globus" key="maxWalltime">00:10:00</profile>
+ <profile namespace="globus" key="userHomeOverride">/lustre/beagle/yadunandb/swiftwork</profile>
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="maxnodes">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="karajan" key="jobThrottle">1.00</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <!-- <workdirectory>/tmp/yadunandb/swiftwork</workdirectory> -->
+ <workdirectory>/lustre/beagle/yadunandb/swiftwork</workdirectory>
+ </pool>
+
+ <pool handle="midway">
+ <execution provider="coaster" jobmanager="local:slurm"/>
+ <profile namespace="globus" key="queue">westmere</profile>
+ <!-- <profile namespace="globus" key="queue">sandyb</profile> -->
+ <!-- <profile namespace="globus" key="queue">amd</profile> -->
+ <profile namespace="globus" key="jobsPerNode">1</profile>
+ <profile namespace="globus" key="maxtime">1200</profile>
+ <profile namespace="globus" key="maxWalltime">00:10:00</profile>
+ <profile namespace="globus" key="highOverAllocation">100</profile>
+ <profile namespace="globus" key="lowOverAllocation">100</profile>
+ <profile namespace="globus" key="slots">4</profile>
+ <profile namespace="globus" key="maxNodes">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="karajan" key="jobThrottle">.64</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <workdirectory>/tmp/yadunand</workdirectory>
+ </pool>
+
+</config>
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/node_uniq.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/node_uniq.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/node_uniq.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+ARGS=$*
+echo ${ARGS[*]} 1>&2
+
+SITE=${ARGS[0]}
+cat $(grep -l $SITE) > tmp
+
+LOCATIONS=($(awk '{ print $1 }' tmp | sort -u))
+
+for LOCATION in ${LOCATIONS[*]}
+do
+ MATCH=$(grep $LOCATION tmp | awk '{print $2}')
+ L=($(echo $LOCATION | sed 's/_/\ /'))
+ echo ${L[0]} ${L[1]} ${MATCH[*]}
+done
+rm tmp;
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/node_uniq.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/reduce_core.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/reduce_core.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/reduce_core.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,8 @@
+reduce(){
+ FILES=($*)
+ DIRNAME=$(dirname ${FILES[0]})
+ RESULT=$DIRNAME/$(hostname -f).$RANDOM.imd
+ awk '{ sum += $1; count += $2 } END { print sum,count }' ${FILES[*]} > $RESULT
+ echo $RESULT
+ rm -rf ${FILES[*]} &> /dev/null
+}
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/reduce_core.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/reducer_template.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/reducer_template.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/reducer_template.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,93 @@
+#!/bin/bash
+# The reducer is expecting to receive filenames as args
+# Each file would contains strings in the format HOSTNAME FILEPATH
+
+# The files passed here are filepointers which need to be resolved
+# before they can be processed by the core reduce function.
+ARGS=($*)
+CMD=${ARGS[0]}
+FILES=(${ARGS[*]:1})
+
+ID=$RANDOM
+HOSTNAME=$(hostname -f)
+if [[ "$HOSTNAME" == *midway* ]]; then # On midway node
+ EXECSERVER=/scratch/midway/yadunand/bin/exec_server
+ LOGFOLDER=/scratch/midway/yadunand
+ FOLDER="/dev/shm"
+ HOSTID="midway_$HOSTNAME"
+ SITE="midway"
+
+elif [[ "$HOSTNAME" == *nid* ]]; then # On beagle node
+ EXECSERVER=/lustre/beagle/yadunandb/bin/exec_server
+ LOGFOLDER=/lustre/beagle/yadunandb/Swift-MapRed/
+ FOLDER="/dev/shm"
+ HOSTID="beagle_$HOSTNAME"
+ SITE="beagle"
+fi
+
+
+# This reduce function must conform to the following specification
+# Takes the CMD type followed by 1 or more files as arguments
+# Returns the content of the file with results
+#USER_REDUCE
+
+fetch()
+{
+ ID=$1
+ TOKEN=$2
+ IFS=$'\ ' PTR=($(echo $TOKEN | sed 's/_/\ /'))
+ REMOTE=${PTR[1]}
+ FILE=${PTR[2]}
+ TARGETFILE=$(basename $FILE)
+ REPODIR=$(dirname $FILE)
+
+ if [ "$REMOTE" == "$HOSTNAME" ]
+ then # Move to an identifiable file if local
+ echo "mv $FILE $REPODIR/$ID.$RANDOM.imd2" 1>&2
+ [ ! -f $FILE ] && echo "$FILE missing" 1>&2
+ mv $FILE $REPODIR/$ID.$RANDOM.imd2
+ else # Fetch to an identifiable file if remote
+ echo "chirp_get $REMOTE $TARGETFILE $REPODIR/$ID.$RANDOM.imd2" 1>&2
+ chirp_get $REMOTE $TARGETFILE $REPODIR/$ID.$RANDOM.imd2
+ fi
+}
+
+plain ()
+{
+ #FILES=($(grep -h "$SITE" $*))
+ ID=$1; shift
+ IFS=$'\r\n' FILES=($(grep -h "$SITE" $*))
+ echo "Matching files : ${FILES[*]}" 1>&2
+ for file_token in ${FILES[*]k}
+ do
+ fetch $ID $file_token &
+ done
+ wait
+ cat $(reduce $FOLDER/$ID*imd2)
+ rm -rf $FOLDER/$ID*imd2 &> /dev/null &
+ exit 0
+}
+
+case "$CMD" in
+ "naive")
+ echo "Using naive reduce" 1>&2
+ cat $(reduce ${FILES[*]})
+ ;;
+ "plain")
+ echo "Using plain reduce" 1>&2
+ plain $ID ${FILES[*]};
+ ;;
+ *)
+ echo "ERROR: Unknown reduction method requested" 1>&2
+ ;;
+esac
+exit 0
+##########################################################################
+TOKEN=($(cat $1))
+FILE=${TOKEN[1]}
+CHIRPDIR=$(dirname $FILE)
+echo "$HOSTNAME $CHIRPDIR $CHIRPDIR/$ID*imd2 " 1>&2
+#cat $(reduce $CHIRPDIR/$ID*imd2)
+echo "$HOSTNAME $(reduce $CHIRPDIR/$ID*imd2)"
+rm -rf $CHIRPDIR/$ID*imd2 &> /dev/null &
+exit 0
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/reducer_template.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/run.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/run.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/run.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,83 @@
+#!/bin/bash
+clear;
+#swift -tc.file tc.data -sites.file sites.xml motoko.swift
+
+#echo "Running run_catsn.swift"
+#swift -tc.file tc.data -sites.file sites.xml run_catsn.swift
+
+
+if [ "$1" != "" ]
+then
+ SITE="$1"
+else
+ SITE="beagle"
+fi
+
+./clean.sh
+
+if [ "$SITE" != "beagle" ]
+then
+ ./compile
+# cp exec_server /lustre/beagle/yadunandb/bin/
+# cp exec_client /lustre/beagle/yadunandb/bin/
+ cp exec_client /scratch/midway/yadunand/bin/
+ cp exec_server /scratch/midway/yadunand/bin/
+ echo "Skipping compile and copy"
+fi;
+#Call as check_error $? <ERR_CODE> <Error message>
+check_error()
+{
+ if [ $1 == $2 ]
+ then
+ echo $3
+ else
+ echo "Run completed with code:$1"
+ fi
+}
+#export COG_OPTS="-Dtcp.channel.log.io.performance=true"
+
+verify()
+{
+ LOOPS=$1
+ RECSIZE=$2
+ ACTUAL_RESULT=($(tail -n 1 final_result))
+ EXPECTED_RESULT=$(($LOOPS * $RECSIZE * 10000 ))
+ if [ "$EXPECTED_RESULT" == "${ACTUAL_RESULT[1]}" ]
+ then
+ echo "---------------------------------------------"
+ echo "COUNT matches expected records"
+ echo "ACTUAL COUNT : ${ACTUAL_RESULT[1]}"
+ echo "EXPECTED COUNT: $EXPECTED_RESULT"
+ echo "---------------------------------------------"
+ else
+ echo "---------------------------------------------"
+ echo "COUNT does *NOT* match expected records"
+ echo "ACTUAL COUNT : ${ACTUAL_RESULT[1]}"
+ echo "EXPECTED COUNT: $EXPECTED_RESULT"
+ cat err_file
+ echo "---------------------------------------------"
+ fi
+}
+
+run_swift_chirp_combiner()
+{
+ LOOPS=$1
+ RECSIZE=$2
+ #./clean.sh &> /dev/null
+ echo "================================================================" >> LAB_RECORDS
+ echo "RUNTYPE :SWIFT+Chirp (With local combiners), SITE:$SITE, CHUNKS:$LOOPS X FILESIZE:$RECSIZE " | tee -a LAB_RECORDS
+ ( time -p swift -tc.file tc.data -config swift.properties -sites.file $SITE.xml swiftreduce.swift -files=$LOOPS -filesize=$RECSIZE ) 2>&1 | tee -a LAB_RECORDS
+ echo "chunks $LOOPS filesize $RECSIZE site $SITE type Swift+Chirp" >> LAB_RECORDS
+ verify $LOOPS $RECSIZE | tee -a LAB_RECORDS
+ echo "================================================================"
+}
+
+if [ "$2" != "" ] && [ "$3" != "" ]
+then
+ run_swift_chirp_combiner $2 $3
+else
+ run_swift_chirp_combiner 20 10
+fi
+
+#run_swift_chirp_combiner 10 10;
+exit 0
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/run.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/server.c
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/server.c (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/server.c 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,181 @@
+/*
+** server.c -- a stream socket server demo
+*/
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <netdb.h>
+#include <arpa/inet.h>
+#include <sys/wait.h>
+#include <signal.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#define PORT "3490" // the port users will be connecting to
+#define MAXDATASIZE 2048
+#define BACKLOG 10 // how many pending connections queue will hold
+
+void sigchld_handler(int s)
+{
+ while(waitpid(-1, NULL, WNOHANG) > 0);
+}
+
+// get sockaddr, IPv4 or IPv6:
+void *get_in_addr(struct sockaddr *sa)
+{
+ if (sa->sa_family == AF_INET) {
+ return &(((struct sockaddr_in*)sa)->sin_addr);
+ }
+
+ return &(((struct sockaddr_in6*)sa)->sin6_addr);
+}
+
+
+int exec_string(char *query, int query_size, char *reply)
+{
+ FILE *p_exec;
+ FILE *f_out;
+ char filename[100];
+ char exec_string[500];
+ int ret;
+
+ sprintf(filename,"/tmp/exec_script_%d", getpid());
+ printf("Debug: filename: %s \n", filename);
+
+ if ( (f_out = fopen(filename,"w+")) == NULL ){
+ printf("Opening[%s] failed: %s\n", filename, strerror(errno));
+ exit(errno);
+ };
+ fwrite(query, query_size, 1, f_out);
+ fclose(f_out);
+
+ sprintf(exec_string, "chmod +x %s; %s",
+ filename, filename);
+ printf("Command string : [%s]\n", exec_string);
+
+ if ( (p_exec = popen(exec_string, "r")) == NULL ){
+ printf("exec_string failed to popen: %s", strerror(errno));
+ return errno;
+ }
+
+ ret = fread(reply, MAXDATASIZE, 1, p_exec);
+ printf("Reply is: \n%s Len = %d\n", reply, (int)strlen(reply));
+ return (strlen(reply));
+}
+
+int main(void)
+{
+ int sockfd, new_fd; // listen on sock_fd, new connection on new_fd
+ struct addrinfo hints, *servinfo, *p;
+ struct sockaddr_storage their_addr; // connector's address information
+ socklen_t sin_size;
+ struct sigaction sa;
+ int yes=1;
+ char s[INET6_ADDRSTRLEN];
+ int rv;
+ char buf[MAXDATASIZE];
+ int numbytes;
+ char *repy;
+
+ memset(&hints, 0, sizeof hints);
+ hints.ai_family = AF_UNSPEC;
+ hints.ai_socktype = SOCK_STREAM;
+ hints.ai_flags = AI_PASSIVE; // use my IP
+
+ if ((rv = getaddrinfo(NULL, PORT, &hints, &servinfo)) != 0) {
+ fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rv));
+ return 1;
+ }
+
+ // loop through all the results and bind to the first we can
+ for(p = servinfo; p != NULL; p = p->ai_next) {
+ if ((sockfd = socket(p->ai_family, p->ai_socktype,
+ p->ai_protocol)) == -1) {
+ perror("server: socket");
+ continue;
+ }
+
+ if (setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &yes,
+ sizeof(int)) == -1) {
+ perror("setsockopt");
+ exit(1);
+ }
+
+ if (bind(sockfd, p->ai_addr, p->ai_addrlen) == -1) {
+ close(sockfd);
+ perror("server: bind");
+ continue;
+ }
+
+ break;
+ }
+
+ if (p == NULL) {
+ fprintf(stderr, "server: failed to bind\n");
+ return 2;
+ }
+
+ freeaddrinfo(servinfo); // all done with this structure
+
+ if (listen(sockfd, BACKLOG) == -1) {
+ perror("listen");
+ exit(1);
+ }
+
+ sa.sa_handler = sigchld_handler; // reap all dead processes
+ sigemptyset(&sa.sa_mask);
+ sa.sa_flags = SA_RESTART;
+ if (sigaction(SIGCHLD, &sa, NULL) == -1) {
+ perror("sigaction");
+ exit(1);
+ }
+
+ printf("server: waiting for connections...\n");
+
+ while(1) { // main accept() loop
+ sin_size = sizeof their_addr;
+ new_fd = accept(sockfd, (struct sockaddr *)&their_addr, &sin_size);
+ if (new_fd == -1) {
+ perror("accept");
+ continue;
+ }
+
+ inet_ntop(their_addr.ss_family,
+ get_in_addr((struct sockaddr *)&their_addr),
+ s, sizeof s);
+ printf("server: got connection from %s\n", s);
+
+ if (!fork()) { // this is the child process
+ int reply_size;
+ char *reply;
+ close(sockfd); // child doesn't need the listener
+ if ((numbytes = recv(new_fd, buf, MAXDATASIZE-1, 0)) == -1) {
+ perror("recv");
+ exit(1);
+ }
+
+ buf[numbytes] = '\0';
+ printf("Server: Received string from client: \n'%s'\n",buf);
+
+ // TODO
+ reply = (char *) malloc (2048);
+ reply_size = exec_string(buf, numbytes, reply);
+ printf("Returning string of size %d \n%s\n", reply_size, reply);
+
+ if (send(new_fd, reply, reply_size, 0) == -1)
+ perror("send");
+ printf("Server: Send string '%s'\n", reply);
+ close(new_fd);
+ free(reply);
+ exit(0);
+ }
+ close(new_fd); // parent doesn't need this
+ }
+ return 0;
+}
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/setup.sh
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/setup.sh (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/setup.sh 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,51 @@
+#!/bin/bash
+
+if [ ! -f "map_core.sh" ]
+then
+ echo "map_core.sh missing"
+ exit 0;
+fi
+
+if [ ! -f "mapper_template.sh" ];
+then
+ echo "mapper_template missing"
+ exit 0;
+else
+ sed -e "/\#USER_MAP/r map_core.sh" mapper_template.sh \
+ > mapper.sh
+ chmod 777 mapper.sh
+fi;
+
+
+
+if [ ! -f "reduce_core.sh" ]
+then
+ echo "reduce_core.sh missing"
+ exit 0;
+fi;
+
+if [ ! -f "reducer_template.sh" ];
+then
+ echo "reducer_template missing"
+ exit 0;
+else
+ sed -e "/\#USER_REDUCE/r reduce_core.sh" reducer_template.sh \
+ > reducer.sh
+ chmod 777 reducer.sh
+fi;
+
+if [ ! -f "combiner_template.sh" ];
+then
+ echo "combiner_template missing"
+ exit 0
+else
+ sed -e "/\#USER_REDUCE/r reduce_core.sh" combiner_template.sh \
+ > combiner.sh
+ chmod 777 combiner.sh
+fi;
+
+
+
+
+
+
Property changes on: SwiftApps/Swift-MapRed/mapred_combiner_V4/setup.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/swift.properties
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/swift.properties (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/swift.properties 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,9 @@
+use.provider.staging=true
+use.wrapper.staging=false
+wrapperlog.always.transfer=true
+execution.retries=0
+lazy.errors=false
+provider.staging.pin.swiftfiles=false
+sitedir.keep=true
+#tcp.port.range=50000,51000
+Dtcp.channel.log.io.performance=true
\ No newline at end of file
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/swiftreduce.swift
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/swiftreduce.swift (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/swiftreduce.swift 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,86 @@
+type file;
+type script;
+// type fileptr will be a regular file to swift, but the contents of
+// which, will be
+// interpreted as globally addressable paths to one or more files.
+// The current format of a fileptr is:
+// SiteX_Node0 /path/to/file0 /path/to/file1 /path/to/file2 ...
+// ...
+// SiteY_Noden /path/to/file0 ...
+type fileptr;
+
+app (fileptr out, file err) map_data (script run, int recsize)
+{
+ bash @run recsize stdout=@out stderr=@err;
+}
+
+app (fileptr out) aggregate (script aggr, fileptr map_results[])
+{
+ bash @aggr @map_results stdout=@out;
+}
+
+int files = @toInt(@arg("files","0"));
+int fsize = @toInt(@arg("filesize","10")); // This would make 10^4 records per file
+
+string sites[] = ["beagle", "midway"];
+
+script mapper <"mapper.sh">;
+script uniq <"node_uniq.sh">;
+script combiner <"combiner.sh">;
+script reducer <"reducer.sh">;
+script deref <"dereference.sh">;
+script aggr <"aggregate.sh">;
+
+fileptr map_out[] <simple_mapper; prefix="map", suffix=".out">;
+file map_err[] <simple_mapper; prefix="map", suffix=".err">;
+// Map stage. File pointers map_out[]
+foreach item,i in [0:files-1] {
+ (map_out[i], map_err[i]) = map_data (mapper, fsize);
+}
+
+// We don't want to send 100s of files to different sites.
+// One fileptr would do
+fileptr all_map <"all_map.out">;
+all_map = aggregate(aggr, map_out);
+
+
+app (file out, file err) reduce_bgl (script reducer, string cmd, fileptr all_map){
+ bgl_bash @reducer cmd @all_map stdout=@out stderr=@err;
+}
+app (file out, file err) reduce_mid (script reducer, string cmd, fileptr all_map){
+ mid_bash @reducer cmd @all_map stdout=@out stderr=@err;
+}
+app (file out, file err) reduce_loc (script reducer, string cmd, file site_results[]){
+ loc_bash @reducer cmd @site_results stdout=@out stderr=@err;
+}
+
+file redsite_out[] <simple_mapper; prefix="reduce", suffix=".out">;
+file redsite_err[] <simple_mapper; prefix="reduce", suffix=".err">;
+
+// Types :
+// Naive : Takes files and gives reduced result, No fileptr hanlding
+// L1/plain : Reduces all results within a site at an internal node
+// L2 : Node -> Site -> Global
+// L3_K_way : Node -> Inter-node -> Site -> Global
+
+/*
+// This is most likely a bug.
+// TODO: Report bug
+foreach site,s in sites{
+ switch (s){
+ case 0: // Beagle
+ (redsite_out[s], redsite_err[s]) = reduce_bgl (reducer, "plain", all_map);
+ case 1: // Midway
+ (redsite_out[s], redsite_err[s]) = reduce_mid (reducer, "plain", all_map);
+ default:
+ tracef("Unknown site: ERROR\n");
+ }
+}
+*/
+
+(redsite_out[0], redsite_err[0]) = reduce_bgl (reducer, "plain", all_map);
+(redsite_out[1], redsite_err[1]) = reduce_mid (reducer, "plain", all_map);
+file final_out <"final_result">;
+file final_err <"final_errors">;
+(final_out, final_err) = reduce_loc (reducer, "naive", redsite_out);
+
Added: SwiftApps/Swift-MapRed/mapred_combiner_V4/tc.data
===================================================================
--- SwiftApps/Swift-MapRed/mapred_combiner_V4/tc.data (rev 0)
+++ SwiftApps/Swift-MapRed/mapred_combiner_V4/tc.data 2013-10-11 03:25:32 UTC (rev 7148)
@@ -0,0 +1,6 @@
+beagle bash /bin/bash null null null
+midway bash /bin/bash null null null
+midway mid_bash /bin/bash null null null
+beagle bgl_bash /bin/bash null null null
+local loc_bash /bin/bash null null null
+local cat /bin/cat null null null
\ No newline at end of file
More information about the Swift-commit
mailing list