[Swift-commit] r2665 - in trunk: . etc libexec resources

[noreply at svn.ci.uchicago.edu]

Author: hategan
Date: 2009-03-09 10:23:13 -0500 (Mon, 09 Mar 2009)
New Revision: 2665

Modified:
   trunk/CHANGES.txt
   trunk/etc/swift.properties
   trunk/libexec/vdl-lib.xml
   trunk/resources/Karajan.stg
Log:
limit number of concurrent iterations in foreach

Modified: trunk/CHANGES.txt
===================================================================
--- trunk/CHANGES.txt	2009-03-09 15:12:24 UTC (rev 2664)
+++ trunk/CHANGES.txt	2009-03-09 15:23:13 UTC (rev 2665)
@@ -1,3 +1,9 @@
+(03/09/09)
+*** Added a throttling parallelFor and changed the swift foreach underlying
+    implementation to it. The throttling parallelFor limits the number of
+    concurrent iterations allowed, which should allow swift to scale better
+    in certain cases.
+
 (02/09/24)
 *** New configuration option wrapper.invocation.mode, specifiable either
     globally in the configuration file or per-site as a profile key,

Modified: trunk/etc/swift.properties
===================================================================
--- trunk/etc/swift.properties	2009-03-09 15:12:24 UTC (rev 2664)
+++ trunk/etc/swift.properties	2009-03-09 15:23:13 UTC (rev 2665)
@@ -302,3 +302,13 @@
 # valid values: absolute, relative
 # wrapper.invocation.mode=absolute
 
+#
+# Limits the number of concurrent iterations that each foreach statement
+# can have at one time. This conserves memory for swift programs that 
+# have large numbers of iterations (which would otherwise all be executed
+# in parallel).
+#
+# Default: 1024
+#
+
+foreach.max.threads=1024

Modified: trunk/libexec/vdl-lib.xml
===================================================================
--- trunk/libexec/vdl-lib.xml	2009-03-09 15:12:24 UTC (rev 2664)
+++ trunk/libexec/vdl-lib.xml	2009-03-09 15:23:13 UTC (rev 2665)
@@ -90,5 +90,7 @@
 	
 	<export name="execute"><elementDef classname="org.griphyn.vdl.karajan.lib.Execute"/></export>
 	<export name="expandArguments"><elementDef classname="org.griphyn.vdl.karajan.lib.ExpandArguments"/></export>
+	
+	<export name="tparallelFor"><elementDef classname="org.griphyn.vdl.karajan.lib.ThrottledParallelFor"/></export>
   </namespace>
 </karajan>

Modified: trunk/resources/Karajan.stg
===================================================================
--- trunk/resources/Karajan.stg	2009-03-09 15:12:24 UTC (rev 2664)
+++ trunk/resources/Karajan.stg	2009-03-09 15:23:13 UTC (rev 2665)
@@ -200,7 +200,7 @@
 >>
 
 foreach(var,in,indexVar,declarations,statements) ::= <<
-<parallelFor name="\$">
+<vdl:tparallelFor name="\$">
   <getarrayiterator>$in$</getarrayiterator>
   <set names="\$\$, $var$">
     <each items="{\$}"/>
@@ -214,7 +214,7 @@
     $if(statements)$
       $parallel(statements=statements)$
     $endif$
-</parallelFor>
+</vdl:tparallelFor>
 >>
 
 call(func, outputs, inputs) ::= <<