[Swift-commit] r4132 - SwiftApps/SwiftR

[noreply at svn.ci.uchicago.edu]

Author: tga
Date: 2011-02-21 18:12:22 -0600 (Mon, 21 Feb 2011)
New Revision: 4132

Modified:
   SwiftApps/SwiftR/IMMEDIATE-TODO
Log:
Updates to TODO list.


Modified: SwiftApps/SwiftR/IMMEDIATE-TODO
===================================================================
--- SwiftApps/SwiftR/IMMEDIATE-TODO	2011-02-22 00:06:43 UTC (rev 4131)
+++ SwiftApps/SwiftR/IMMEDIATE-TODO	2011-02-22 00:12:22 UTC (rev 4132)
@@ -1,7 +1,4 @@
-1) swift bcast if exportAll files/data and/or xmit this via prov staging w/ caching of duplicates
 
-2) Fast branch: working? faster?
-
 3) speed: where is bottleneck?  how to measure and tune?
 - ParallelCI and ParallelBoostrap benchmarks
 - smaller focused micro-tests to create plots of speed vs param size and efficiency vs runtime of the evaluated function.
@@ -11,33 +8,24 @@
 - test on Ranger
 - get this to Tim Bates to test on Eddie ()
 
-5) Timing: dramatic bencmark results for new proposal.
 
 
 VERY HIGH:
--- Benchmarks for grant app - grant app is due 21 Feb
--- New results by around 11 Feb
-    - Synthetic benchmark
+-- Benchmarks 
+    Timing: dramatic bencmark results for new proposal.
     - Parallel bootstrap
     - Parallel CI
-    - smaller focused micro-tests to create plots of speed vs param size and efficiency vs runtime of the evaluated function.
+    - smaller focused micro-tests looking at param size
     - Sarah Kenny FMRI
     - parallel OpenMx Tests
--- Candidate systems:
- - Eureka/cobalt 
- - Beagle? (Prelim: crays crow)
- - Sicortex (prob. not)
- - BG/P
--- Benchmarks on average user machines
+    - Candidate systems:
+        - Eureka/cobalt - need to finish up
+        - Beagle - might be working
+        - Sicortex (NO)
+        - BG/P (NO)
 
-HIGH:
--- Test pbsf (on PADS?)
 
 HIGH:
-in swift.properties we should make sure we dont return info logs we should make sure that the amount of logging on both the worker side and the client side is as low as it can go
-* Turn off bashrc processing, etc shell script
-
-HIGH:
 - tests
 -- simple calibration tests (n args, arg sizes, durations, etc)
 -- openmx tests
@@ -81,6 +69,17 @@
 * Cleanup ssh worker processes: add a watchdog that detects when worker.pl has 
     gone away
 
+MID:
+* monitor batch jobs to make sure they launch ok and keep the user in the loop
+* restart batch jobs automatically if time runs out
+
+MID:
+* Fast branch: working? faster?
+
+MID:
+Investigate use of swift broadcase:
+    swift bcast if exportAll files/data and/or xmit this via prov staging w/ caching of duplicates
+
 LOW: (unless needed by immediate OpenMx app or test)
 - complete sf compat functions (sapply, lapply -> for openMx, based on usage)
 
@@ -108,3 +107,16 @@
     - each request provides a reply channel
     - swift server forks off a worker thread for
         each request
+
+=========================================================================
+Completed:
+=========================================================================
+HIGH:
+-- Test pbsf (on PADS?)
+
+HIGH:
+-- Check job throttling setting
+* Turn off bashrc processing, etc shell script
+
+HIGH:
+in swift.properties we should make sure we dont return info logs we should make sure that the amount of logging on both the worker side and the client side is as low as it can go