From davidk at ci.uchicago.edu  Tue Jun  5 11:45:59 2012
From: davidk at ci.uchicago.edu (davidk at ci.uchicago.edu)
Date: Tue,  5 Jun 2012 11:45:59 -0500 (CDT)
Subject: [Swift-commit] r5803 - trunk/bin
Message-ID: <20120605164559.11AA89CD0E@svn.ci.uchicago.edu>

Author: davidk
Date: 2012-06-05 11:45:58 -0500 (Tue, 05 Jun 2012)
New Revision: 5803

Modified:
   trunk/bin/start-coaster-service
Log:
Ability to allocate multiple cores per task with condor - useful for MPI jobs


Modified: trunk/bin/start-coaster-service
===================================================================
--- trunk/bin/start-coaster-service	2012-05-30 18:58:21 UTC (rev 5802)
+++ trunk/bin/start-coaster-service	2012-06-05 16:45:58 UTC (rev 5803)
@@ -271,14 +271,18 @@
       crash "CONDOR_WORKERS undefined - please check coaster-service.conf"
    fi
 
+   if [ -z "$CONDOR_MACHINE_COUNT" ]; then
+      export CONDOR_MACHINE_COUNT=1
+   fi
+
 cat <<EOF > start_workers.submit
 output = workers.stdout
 error = workers.stderr
-#initialdir = \$PWD 
 executable = $SWIFT_BIN/$WORKER
 arguments = $EXECUTION_URL node .
 notification = Never
 leave_in_queue = TRUE
+machine_count = $CONDOR_MACHINE_COUNT
 queue $CONDOR_WORKERS
 EOF
 



From swift at ci.uchicago.edu  Wed Jun  6 18:10:03 2012
From: swift at ci.uchicago.edu (swift at ci.uchicago.edu)
Date: Wed,  6 Jun 2012 18:10:03 -0500 (CDT)
Subject: [Swift-commit] cog r3421
Message-ID: <20120606231004.833A08D00D62@bridled.ci.uchicago.edu>

------------------------------------------------------------------------
r3421 | davidkelly999 | 2012-06-06 17:31:37 -0500 (Wed, 06 Jun 2012) | 5 lines

Coaster port assignments were not honoring GLOBUS_TCP_PORT_RANGE after the 690 fixes 
This was causing issues when not specifying ports with persistent coasters
I believe it was also affecting bootstrapping
 

------------------------------------------------------------------------
Index: modules/provider-coaster/src/org/globus/cog/abstraction/coaster/service/LocalTCPService.java
===================================================================
--- modules/provider-coaster/src/org/globus/cog/abstraction/coaster/service/LocalTCPService.java	(revision 3420)
+++ modules/provider-coaster/src/org/globus/cog/abstraction/coaster/service/LocalTCPService.java	(working copy)
@@ -32,6 +32,7 @@
 import org.globus.cog.karajan.workflow.service.channels.KarajanChannel;
 import org.globus.cog.karajan.workflow.service.channels.TCPChannel;
 import org.globus.common.CoGProperties;
+import org.globus.net.PortRange;
 
 public class LocalTCPService implements Registering, Service, Runnable {
     public static final Logger logger = Logger.getLogger(LocalTCPService.class);
@@ -100,6 +101,10 @@
         try {
             channel = ServerSocketChannel.open();
             channel.configureBlocking(true);
+            if(port == 0) {
+              PortRange portRange = PortRange.getTcpInstance();
+              port = portRange.getFreePort(port);
+            }
             channel.socket().bind(new InetSocketAddress(port));
             
             if (serverThread == null) {


From wilde at ci.uchicago.edu  Fri Jun  8 13:10:40 2012
From: wilde at ci.uchicago.edu (wilde at ci.uchicago.edu)
Date: Fri,  8 Jun 2012 13:10:40 -0500 (CDT)
Subject: [Swift-commit] r5804 - in SwiftApps/ParVis/HiRAMTools: . JUNK
Message-ID: <20120608181040.D68C09CC99@svn.ci.uchicago.edu>

Author: wilde
Date: 2012-06-08 13:10:40 -0500 (Fri, 08 Jun 2012)
New Revision: 5804

Added:
   SwiftApps/ParVis/HiRAMTools/JUNK/
   SwiftApps/ParVis/HiRAMTools/JUNK/checkgroups.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/combine.chk.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/combine.orig.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/find_anomalies.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/get_atmos_scalar.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copy.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copyL.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/makeyearly-cdo-fail.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/makeyearly.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/run.combine_realization.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/runall.combine_realizations.config
   SwiftApps/ParVis/HiRAMTools/JUNK/runall.combine_realizations.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/runall4.combine_realizations.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/runcombine.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/test-makeyearly-cdo.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/testfailures.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/y1.sh
   SwiftApps/ParVis/HiRAMTools/combine.sh
   SwiftApps/ParVis/HiRAMTools/combine_realization.sh
   SwiftApps/ParVis/HiRAMTools/makeyearly-cdo.sh
   SwiftApps/ParVis/HiRAMTools/makeyearly_realization.sh
   SwiftApps/ParVis/HiRAMTools/runall.sh
Removed:
   SwiftApps/ParVis/HiRAMTools/checkgroups.sh
   SwiftApps/ParVis/HiRAMTools/combine.chk.sh
   SwiftApps/ParVis/HiRAMTools/combine.orig.sh
   SwiftApps/ParVis/HiRAMTools/combine.sh
   SwiftApps/ParVis/HiRAMTools/combine_realization.sh
   SwiftApps/ParVis/HiRAMTools/find_anomalies.sh
   SwiftApps/ParVis/HiRAMTools/get_atmos_scalar.sh
   SwiftApps/ParVis/HiRAMTools/get_real_copy.sh
   SwiftApps/ParVis/HiRAMTools/get_real_copyL.sh
   SwiftApps/ParVis/HiRAMTools/makeyearly-cdo.sh
   SwiftApps/ParVis/HiRAMTools/makeyearly.sh
   SwiftApps/ParVis/HiRAMTools/makeyearly_realization.sh
   SwiftApps/ParVis/HiRAMTools/run.combine_realization.sh
   SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.config
   SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.sh
   SwiftApps/ParVis/HiRAMTools/runall.sh
   SwiftApps/ParVis/HiRAMTools/runall4.combine_realizations.sh
   SwiftApps/ParVis/HiRAMTools/runcombine.sh
   SwiftApps/ParVis/HiRAMTools/y1.sh
Modified:
   SwiftApps/ParVis/HiRAMTools/README
   SwiftApps/ParVis/HiRAMTools/runall.config
Log:
Cleanup and documentation of all scripts.

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/checkgroups.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/checkgroups.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/checkgroups.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/checkgroups.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,21 @@
+#! /bin/bash
+
+# /intrepid-fs0/users/lzamboni/persistent/en1eo8Ic1/history/19500101/atmos_4xdaily.tile1.nc
+
+sed -e 's/^.*history.//' groups | sort >todo
+find /intrepid-fs0/users/wilde/persistent/en1eo8Ic1/combined/run016 -type f -print | fgrep .nc | sed -e 's/^.*run016.//' | sort >done
+
+echo
+echo diff: "<todo >done"
+echo
+diff todo done
+
+echo
+echo last 20 files produced were:
+echo
+ls -ltd /intrepid-fs0/users/wilde/persistent/en1eo8Ic1/combined/run016/*/* | head -20
+
+echo
+echo File lengths:
+echo
+ls -ltd /intrepid-fs0/users/wilde/persistent/en1eo8Ic1/combined/run016/*/* | awk '{print $5}' | sort | uniq -c

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/combine.chk.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/combine.chk.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/combine.chk.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/combine.chk.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,33 @@
+#! /bin/bash
+
+set -o errexit
+
+mkdir -p /scratch/$USER
+
+tmp=$(mktemp -d /scratch/$USER/combine_XXXXXX)
+
+echo mktemp error code: $?
+
+cd $tmp
+
+outdir=$1
+inpattern=$2
+date=$(basename $(dirname $inpattern))
+basepattern=$(basename $inpattern)
+
+infiles="$2*"
+for f in $infiles; do
+  dd if=$f of=$(basename $f) bs=4M
+done
+
+/home/ckerr/bin/fms/tools/x86_64/bin/mppnccombine.exe out.nc ${basepattern}*
+
+mkdir -p $outdir/$date
+dd if=out.nc of=$outdir/$date/$basepattern.trans bs=4M
+mv $outdir/$date/$basepattern.trans $outdir/$date/$basepattern
+
+cd ..
+rm -rf $tmp
+
+
+

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/combine.orig.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/combine.orig.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/combine.orig.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/combine.orig.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,28 @@
+#! /bin/bash
+
+set -x 
+
+mkdir -p /scratch/wilde
+tmp=$(mktemp -d /scratch/wilde/combine_XXXX)
+cd $tmp
+
+outdir=$1
+inpattern=$2
+date=$(basename $(dirname $inpattern))
+basepattern=$(basename $inpattern)
+
+infiles="$2*"
+for f in $infiles; do
+  dd if=$f of=$(basename $f) bs=4M
+done
+
+/home/ckerr/bin/fms/tools/x86_64/bin/mppnccombine.exe out.nc ${basepattern}*
+
+mkdir -p $outdir/$date
+dd if=out.nc of=$outdir/$date/$basepattern bs=4M
+
+cd ..
+rm -rf $tmp
+
+
+

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/find_anomalies.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/find_anomalies.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/find_anomalies.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/find_anomalies.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,18 @@
+#! /bin/sh
+
+reals=/intrepid-fs0/users/wilde/persistent/LZ/combined
+cd $reals
+
+for real in *; do
+  echo Realization: $real
+  ( cd $real
+    for y in *; do
+      echo "  Year: " $y
+      ( cd $y
+        for f in *.nc; do
+          echo -n $f; ncdump -h $f | grep 'time =' || echo
+        done | awk '{print "    ",$1,$7}' | grep 'atmos.*\.tile[0-9].* (1' | grep -v scalar
+      )
+    done
+  )
+done

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/get_atmos_scalar.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/get_atmos_scalar.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/get_atmos_scalar.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/get_atmos_scalar.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,17 @@
+#! /bin/bash
+
+src=/intrepid-fs0/users/lzamboni/persistent
+dest=/intrepid-fs0/users/wilde/persistent/LZ/combined
+
+f=atmos_scalar.nc
+
+reals=$(cat /home/wilde/LZ/realizations)
+
+for r in $reals; do
+  cd $src/$r/history
+  years=$( /bin/ls -1d [0-9]??????? )
+  for y in $years; do
+    cp $y/$f $dest/$r/$y
+  done
+done
+

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copy.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/get_real_copy.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copy.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copy.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,26 @@
+#! /bin/bash
+
+# src=/intrepid-fs0/users/lzamboni/persistent
+
+src=/intrepid-fs0/users/wilde/persistent/LZ/combined
+dest=$PWD
+
+real=$1
+
+if [ _$1 = _ ]; then
+  echo realization to copy was not specified - exiting; exit 1
+fi
+
+if [ -e _$1 ]; then
+  echo $1 already exits - exiting; exit 1
+fi
+
+cd $src/$real
+dirs=$(find * -type d -print)
+
+for d in $dirs; do
+  mkdir -p $dest/$real/$d
+  for f in $d/*.nc; do
+    ln -s $PWD/$f $dest/$real/$f
+  done
+done

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copyL.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/get_real_copyL.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copyL.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copyL.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,26 @@
+#! /bin/bash
+
+# src=/intrepid-fs0/users/lzamboni/persistent
+
+src=/intrepid-fs0/users/wilde/persistent/LZ/combined
+dest=$PWD
+
+real=$1
+
+if [ _$1 = _ ]; then
+  echo realization to copy was not specified - exiting; exit 1
+fi
+
+if [ -e _$1 ]; then
+  echo $1 already exits - exiting; exit 1
+fi
+
+cd $src/$real
+dirs=$(find * -type d -print)
+
+for d in $dirs; do
+  mkdir -p $dest/$real/history/$d
+  for f in $d/*.nc; do
+    ln -s $PWD/$f $dest/$real/history/$f
+  done
+done

Added: SwiftApps/ParVis/HiRAMTools/JUNK/makeyearly-cdo-fail.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/makeyearly-cdo-fail.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/makeyearly-cdo-fail.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,13 @@
+#! /bin/bash
+
+set -o errexit
+
+# Create a fast (local-filesystem) working dir
+
+echo About to run failing mkdir
+
+mkdir -p /NOscratch/wilde
+
+echo Should not get here!
+exit 42
+


Property changes on: SwiftApps/ParVis/HiRAMTools/JUNK/makeyearly-cdo-fail.sh
___________________________________________________________________
Added: svn:executable
   + *

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/makeyearly.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/makeyearly.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/makeyearly.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/makeyearly.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,231 @@
+#! /bin/bash
+
+set -x
+
+infile=$1  # must be full path
+outdir=$2  # must be full path
+
+realdir=$(dirname $(dirname $1))  # realization dir, e.g., en*
+realname=$(basename $realdir)      # realization name
+f=$(basename $1)
+d1=$(dirname $1)
+yfirst=$(echo $(basename $d1) | sed -e 's/....$//')
+ymid=$((yfirst+2))
+#ydir1=${yfirst}0101
+d2=$realdir/${ymid}0701
+
+period=$(basename $f .nc | sed -e 's/^.*_//' -e 's/\..*$//' -e 's/_.*//')
+
+leap()
+{
+  case $1 in
+    1952|1956|1960|1964|1968|1972|1976|1980|1984|1988|1992|1996|2000) echo 1 ;;
+    *) echo 0 ;;
+  esac
+}
+
+units()
+{
+  p=$1 # period
+  y=$2 # year
+  case $period in
+    4xdaily)   echo $((4*(365+$(leap $y)))) ;;
+    8xdaily)   echo $((8*(365+$(leap $y)))) ;;
+    daily)     echo $((1*(365+$(leap $y)))) ;;
+    month)     echo 12                     ;;
+    * )
+  esac
+}
+
+mkdir -p /scratch/wilde
+tmp=$(mktemp -d /scratch/wilde/makeyearly_XXXXXX)
+startdir=$PWD
+cd $tmp
+
+export PATH=$PATH:/soft/apps/nco-3.9.9/bin
+# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/soft/apps/nco-3.9.9/lib
+
+o=$outdir/$realname
+
+yd1=$o/$((yfirst+0))0101
+yd2=$o/$((yfirst+1))0101
+yd3=$o/$((yfirst+2))0101
+yd4=$o/$((yfirst+3))0101
+yd5=$o/$((yfirst+4))0101
+
+y1=$((yfirst+0))
+y2=$((yfirst+1))
+y3=$((yfirst+2))
+y4=$((yfirst+3))
+y5=$((yfirst+4))
+
+set +x
+e=echo
+
+$e mkdir -p $yd1 $yd2 $yd3 $yd4 $yd5
+
+$e dd if=$d1/$f of=$f bs=4M           # copy in first 2.5 year fle
+
+su=1                                  # start unit
+u1=$(units $period $y1)
+eu=$u1                                # end unit
+
+$e ncks -F -d time,$su,$eu $f out.nc   # extract year 1
+$e dd if=out.nc of=$yd1/$f bs=4M
+$e rm out.nc
+
+su=$((eu+1))
+u2=$(units $period $y2)
+eu=$((eu+$u2))
+
+$e ncks -F -d time,$su,$eu $f out.nc  # extract year 2
+$e dd if=out.nc of=$yd2/$f bs=4M
+$e rm out.nc
+
+su=$((eu+1))
+u3=$(units $period $y3)
+heu=$((u3/2))
+eu=$((eu+$heu))
+
+$e ncks -F -d time,$su,$eu $f tmp.a.nc # extract year 3 part a
+$e dd if=$d2/$f of=$f bs=4M            # copy in second 2.y year file
+
+su=1
+eu=$heu
+
+$e ncks -F -d time,$su,$eu $f tmp.b.nc # extract year 3 part b
+
+$e ncrcat tmp.a.nc tmp.b.nc out.nc     # merge year3 parts a and b
+$e dd if=out.nc of=$yd3/$f bs=4M
+$e rm out.nc tmp.a.nc tmp.b.nc
+
+su=$((eu+1))
+u4=$(units $period $y4)
+eu=$((eu+$u4))
+
+$e ncks -F -d time,$su,$eu $f out.nc   # extract year 4
+$e dd if=out.nc of=$yd4/$f bs=4M
+$e rm out.nc
+
+su=$((eu+1))
+u5=$(units $period $y5)
+eu=$((eu+$u4))
+
+$e ncks -F -d time,$su,$eu $f out.nc  # extract year 5
+$e dd if=out.nc of=$yd5/$f bs=4M
+$e rm out.nc $f
+
+$e cd $startdir
+rm -rf $tmp
+
+exit
+
+######################
+#
+
+the origin of the problem is that in the first 6 month of the year (Jan-Jun)
+there are less days than in the second half of the year (Jul-Dec), and that
+2.5 years refers to the # of months.
+
+The difference is 3 days in a normal year, 2 in a leap year.
+
+So the variations are not simply due by the presence of a leap year. These are
+the categories.  Let me define these before:
+ 
+# normal= no leap in the 2.5 directory.
+# 1st half=directoryName is yyyy0101
+# 2nd half=directoryName is yyyy0701
+# 
+# N1 normal, 1st half -->3644 steps (911 days*4)
+# N2 normal, 2nd half -->3656 steps (914 days*4)
+# L1 leap, 1st half -->3648 steps   (912 days*4)
+# L2 leap, 2nd half -->3660 steps.  (915 days*4)
+
+# leap: 1952 1956 1960 1964 1968 1972 1976 1980 1984 1988 1992 1996 2000
+
+# Jan, Feb, Mar, Apr, May, June = (31+28+31+30+31+30) = 181
+
+# N-year: 181+184 x4= 724 + 736 = 1460
+# L-year: 182+184 x4= 728 + 736 = 1474
+
+
+19500101 1   L1: 3648 
+1951     1
+1952     / L
+
+19520701 /   N2: 3656
+1953     1
+1954     1
+----
+19550101 1   L1: 3648
+1956     1 L
+1957     /
+
+19570701 +   N2: 3656
+1958     1
+1959     1
+----
+19600101 1 L L1: 3648
+1961     1
+1962     /
+
+19620701 +   L2: 3660
+1963     1
+1964     1 L
+----
+19650101 1   N1: 3648
+1966     1
+1967     /
+
+19670701 +   L2: 3660
+1968     1 L
+1969     1
+----
+19700101 1   L1: 3648
+1971     1
+1972     / L
+
+19720701 +   N2: 3656
+1973     1
+1974     1
+----
+19750101 1   L1: 3648
+1976     1 L
+1977     /
+
+19770701 +   N2: 3656
+1978     1
+1979     1
+----
+19800101 1 L L1: 3648
+1981     1
+1982     /
+
+19820701 +   L2: 3660
+1983     1
+1984     1 L
+----
+19850101 1   N1: 3648
+1986     1
+1987     /
+
+19870701 +   L2: 3660
+1988     1 L
+1989     1
+----
+19900101 1   L1: 3648
+1991     1
+1992     / L
+
+19920701 +   N2: 3656
+1993     1
+1994     1
+----
+19950101 1   L1: 3648
+1995     1
+1996     / L
+
+19970701 +   N2: 3656
+1998     1
+1999     1
+----

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/run.combine_realization.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/run.combine_realization.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/run.combine_realization.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/run.combine_realization.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,5 @@
+#! /bin/sh
+
+./combine_realization.sh  \
+  /intrepid-fs0/users/lzamboni/persistent/en1eo8Ic1/history/19 \
+  /intrepid-fs0/users/wilde/persistent/en1eo8Ic1/combined/ 

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/runall.combine_realizations.config (from rev 5750, SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.config)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/runall.combine_realizations.config	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/runall.combine_realizations.config	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,4 @@
+realdir=/intrepid-fs0/users/lzamboni/persistent           # Base dir of realizations
+realtodo=real.todo                                        # File of names of realization subdirs to process
+realdone=real.done                                        # File of names of realization subdirs completed
+outdir=/intrepid-fs0/users/wilde/persistent/LZ/combined4  # Base dir for combined output

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/runall.combine_realizations.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/runall.combine_realizations.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/runall.combine_realizations.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,50 @@
+#! /bin/sh
+
+# ls -1d en*/history/[0-9][0-9][0-9][0-9]????   eur$ ls -1d en*/history/[0-9][0-9][0-9][0-9]????
+
+# The following can be made into parameters to this script. Set here for now.
+
+realdir=/intrepid-fs0/users/lzamboni/persistent          # Base dir of realizations
+realtodo=real.todo                                       # File of names of realization subdirs to process
+realdone=real.done                                       # File of names of realization subdirs completed
+outdir=/intrepid-fs0/users/wilde/persistent/LZ/combined  # Base dir for combined output
+
+source ./runall.combine_realizations.config              # can overwrite above config params
+
+# Helpful patterns for filename matching
+
+yyyy="[0-9][0-9][0-9][0-9]"
+yyyymmdd="${yyyy}[0-9][0-9][0-9][0-9]"
+
+mkdir -p $outdir
+
+while true; do
+
+  real=$(diff $realtodo $realdone | grep '^<' | sed -e 's/^..//' | head -1) # next realization to do
+  if [ _$real = _ ]; then
+    break
+  fi
+
+  echo Combining $real
+
+  find $realdir/$real/history/$yyyymmdd  |
+    grep .nc.$yyyy      |
+    sed -e 's/.....$//' | sort | uniq > $real.groups
+
+  echo Created $real.groups
+
+  echo  ./combine_realization.sh    \
+    "$realdir/$real/history/$yyyymmdd" \
+    $outdir/$real 
+
+  ./combine_realization.sh    \
+    "$realdir/$real/history/$yyyymmdd" \
+    $outdir/$real 
+
+  echo Completed $real
+  echo $real >>$realdone
+
+done
+
+echo All combines are done.
+

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/runall4.combine_realizations.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/runall4.combine_realizations.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/runall4.combine_realizations.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/runall4.combine_realizations.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,48 @@
+#! /bin/sh
+
+# ls -1d en*/history/[0-9][0-9][0-9][0-9]????   eur$ ls -1d en*/history/[0-9][0-9][0-9][0-9]????
+
+# The following can be made into parameters to this script. Set here for now.
+
+realdir=/intrepid-fs0/users/lzamboni/persistent           # Base dir of realizations
+realtodo=real4.todo                                       # File of names of realization subdirs to process
+realdone=real4.done                                       # File of names of realization subdirs completed
+outdir=/intrepid-fs0/users/wilde/persistent/LZ/combined4  # Base dir for combined output
+
+# Helpful patterns for filename matching
+
+yyyy="[0-9][0-9][0-9][0-9]"
+yyyymmdd="${yyyy}[0-9][0-9][0-9][0-9]"
+
+mkdir -p $outdir
+
+while true; do
+
+  real=$(diff $realtodo $realdone | grep '^<' | sed -e 's/^..//' | head -1) # next realization to do
+  if [ _$real = _ ]; then
+    break
+  fi
+
+  echo Combining $real
+
+  find $realdir/$real/history/$yyyymmdd  |
+    grep .nc.$yyyy      |
+    sed -e 's/.....$//' | sort | uniq > $real.groups
+
+  echo Created $real.groups
+
+  echo  ./combine_realization.sh    \
+    "$realdir/$real/history/$yyyymmdd" \
+    $outdir/$real 
+
+  ./combine_realization.sh    \
+    "$realdir/$real/history/$yyyymmdd" \
+    $outdir/$real 
+
+  echo Completed $real
+  echo $real >>$realdone
+
+done
+
+echo All combines are done.
+

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/runcombine.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/runcombine.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/runcombine.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/runcombine.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,5 @@
+#! /bin/sh
+
+$HOME/LZ/combine.csh \
+    -a /intrepid-fs0/users/lzamboni/persistent/en1eo8Ic1/history \
+    -s 19500101 -n 0 -m 30 # -q

Added: SwiftApps/ParVis/HiRAMTools/JUNK/test-makeyearly-cdo.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/test-makeyearly-cdo.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/test-makeyearly-cdo.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,32 @@
+# ./makeyearly-cdo.sh /intrepid-fs0/users/wilde/persistent/LZ/combined/en1eo12Ic1/19500101/grid_spec.tile4.nc \
+#                    /intrepid-fs0/users/wilde/persistent/LZ/combined/en1eo12Ic1/19520701/grid_spec.tile4.nc
+
+#./makeyearly-cdo.sh /intrepid-fs0/users/wilde/persistent/LZ/combined/en1eo12Ic1/19500101/atmos_8xdaily_instant.tile3.nc \
+#                    /intrepid-fs0/users/wilde/persistent/LZ/combined/en1eo12Ic1/19520701/atmos_8xdaily_instant.tile3.nc
+
+#./makeyearly-cdo.sh /intrepid-fs0/users/lzamboni/persistent/combined/Climo/run002/19500101/atmos_month.tile1.nc \
+#                    /intrepid-fs0/users/lzamboni/persistent/combined/Climo/run002/19520701/atmos_month.tile1.nc
+
+#./makeyearly-cdo.sh /intrepid-fs0/users/lzamboni/persistent/combined/Climo2/run004/19500101/atmos_daily.tile1.nc \
+#                    /intrepid-fs0/users/lzamboni/persistent/combined/Climo2/run004/19520701/atmos_daily.tile1.nc \
+#                    /home/wilde/LZ/tests/y2
+
+#./makeyearly-cdo.sh /intrepid-fs0/users/lzamboni/persistent/combined/Climo2/run004/19500101/atmos_scalar.nc \
+#                    /intrepid-fs0/users/lzamboni/persistent/combined/Climo2/run004/19520701/atmos_scalar.nc \
+#                    /home/wilde/LZ/tests/y2
+
+./makeyearly-cdo.sh /intrepid-fs0/users/lzamboni/persistent/combined/Climo2/run004/19500101/grid_spec.tile1.nc \
+                    /intrepid-fs0/users/lzamboni/persistent/combined/Climo2/run004/19520701/grid_spec.tile1.nc \
+                    /home/wilde/LZ/tests/y2
+
+
+
+
+
+
+
+
+
+
+
+

Added: SwiftApps/ParVis/HiRAMTools/JUNK/testfailures.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/testfailures.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/testfailures.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,98 @@
+#! /bin/bash
+
+cdir=$1     # combined dir
+outbase=$2  # base of outdir for equivalent yearly files
+
+bindir=$(dirname $0)
+bindir=$(cd $bindir; pwd)
+
+echo bindir=$bindir
+
+# Enable one of the following patterns to name the output dir:
+
+outsuffix=$(basename $cdir)                              # rname
+outsuffix=$(basename $(dirname $cdir))                   # rname/rundir, use rname
+outsuffix=$(basename $(dirname $cdir))/$(basename $cdir) # rname/rundir, use both
+
+# generate the list of files to process: do every half-decade
+
+echo f1 f2 >ncfiles
+echo f1 f2 >>ncfiles
+
+# for y in $(seq 1950 5 1997); do
+#   startyear=${y}0101; 
+#   nextyear=$((y+2))0701
+#   find $cdir/$startyear -type f | fgrep .nc | sed -e "s,^\\(.*\\)/$startyear/\\(.*\\),\\1/$startyear/\\2 \\1/$nextyear/\\2 $outdir,"
+# done >>ncfiles
+
+# Create new runNNN directory
+
+rundir=$( echo run??? | sed -e 's/^.*run//' | awk '{ printf("run%03d\n", $1+1)}' )
+mkdir $rundir
+
+# Generate Swift config files (cf, tc, sites.xml):
+
+cat >$rundir/cf <<END
+
+wrapperlog.always.transfer=true
+sitedir.keep=true
+execution.retries=10
+lazy.errors=true
+status.mode=provider
+
+END
+
+cat >$rundir/sites.xml <<END
+
+<config>
+
+   <pool handle="local">
+     <execution provider="local" />
+     <profile namespace="karajan" key="jobThrottle">.01</profile>
+     <profile namespace="karajan" key="initialScore">10000</profile>
+     <filesystem provider="local"/>
+     <workdirectory>$PWD/swiftwork</workdirectory>
+   </pool>
+
+  <pool handle="cobalt">
+    <execution provider="coaster" jobmanager="local:cobalt" url="eureka.alcf.anl.gov" />
+    <filesystem provider="local" />
+    <!-- <profile namespace="globus"key="internalHostname">172.17.3.11</profile> -->
+    <profile namespace="globus"  key="project">MTCScienceApps</profile>
+    <profile namespace="globus"  key="queue">default</profile>
+    <profile namespace="karajan" key="jobthrottle">2.56</profile>
+    <profile namespace="karajan" key="initialScore">10000</profile>
+    <profile namespace="globus"  key="jobsPerNode">8</profile>
+    <!-- <profile namespace="globus"  key="workerLoggingLevel">DEBUG</profile> -->
+    <!-- <profile namespace="globus"  key="workerLoggingDirectory">/home/jonmon/Workspace/Swift/tests/catsnsleep</profile> -->
+    <profile namespace="globus"  key="slots">32</profile>
+    <profile namespace="globus"  key="maxTime">28000</profile> <!-- ~ 8hrs -->
+    <profile namespace="globus"  key="maxWallTime">00:20:00</profile>
+    <profile namespace="globus"  key="lowoverallocation">100</profile>
+    <profile namespace="globus"  key="highoverallocation">100</profile>
+    <profile namespace="globus"  key="nodeGranularity">1</profile>
+    <profile namespace="globus"  key="maxNodes">1</profile>
+    <workdirectory>$PWD/$rundir/swiftwork</workdirectory>
+  </pool>
+
+</config>
+
+END
+
+cat >$rundir/tc <<END
+
+cobalt makeyearly $bindir/makeyearly-cdo-fail.sh
+
+END
+
+# Place file list and swift script in run dir, set params, make output dir, and run script
+
+cp ncfiles $bindir/makeyearly.swift $rundir
+
+cd $rundir
+echo Running in directory $rundir
+runid=$(basename $rundir) # un-needed?
+outdir=$outbase/$outsuffix/$runid
+mkdir -p $outdir
+
+swift -config cf -tc.file tc -sites.file sites.xml makeyearly.swift -rundir=$outdir -ncfiles=ncfiles >& swift.out


Property changes on: SwiftApps/ParVis/HiRAMTools/JUNK/testfailures.sh
___________________________________________________________________
Added: svn:executable
   + *

Copied: SwiftApps/ParVis/HiRAMTools/JUNK/y1.sh (from rev 5750, SwiftApps/ParVis/HiRAMTools/y1.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/y1.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/y1.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,29 @@
+#! /bin/sh
+
+export PATH=$PATH:/soft/apps/nco-3.9.9/bin
+# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/soft/apps/nco-3.9.9/lib
+
+d1=19500101
+d2=19520701
+f=ice_month.nc
+
+o=testyears
+y1=$o/19500101
+y2=$o/19510101
+y3=$o/19520101
+y4=$o/19530101
+y5=$o/19540101
+
+mkdir -p $y1 $y2 $y3 $y4 $y5
+
+ncks -F -d time,1,12    $d1/$f $y1/$f
+
+ncks -F -d time,13,24   $d1/$f $y2/$f
+
+ncks -F -d time,25,30   $d1/$f $y3/$f.a
+ncks -F -d time,1,6     $d2/$f $y3/$f.b
+ncrcat $y3/$f.a $y3/$f.b       $y3/$f
+
+ncks -F -d time,7,18    $d2/$f $y4/$f
+
+ncks -F -d time,19,30   $d2/$f $y5/$f

Modified: SwiftApps/ParVis/HiRAMTools/README
===================================================================
--- SwiftApps/ParVis/HiRAMTools/README	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/README	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,24 +1,84 @@
 
-1. Add Swift to your path. For now we will use a Swift version maintained by
-   Swift team member Jon Monette:
+The top-level script in this toolkit is "runall.sh". This is the only script
+that you should need to run directly.
 
-PATH=/home/jonmon/Library/Swift/swift-0.93/bin:/home/jonmon/Library/jdk1.7.0_03/bin:$PATH
+runall.sh takes a config file which specifies what realization processing
+script to run, where to find the input realizations, and where to place the
+processed output realizations.
 
-2. Edit runall.combine_realizations.config
+  makeyearly_realization.sh  => annualizes a realization
+  combine_realization.sh     => combines a realization
 
-3. Place the list of realizations to process in a file named "real.todo" and
-   create and empty file "real.done", like so (for example, to process 2
-   realizations, Climo_001 and Climo_023):
+The script calling structure is:
 
-cat >real.todo
-Climo_001
-Climo_023
-^D
+  runall.sh   # called by the user, runs the scripts below
 
-cat >real.done
-^D
+    makeyearly_realization.sh
+      makeyearly.swift
+        makeyearly-cdo.sh (app, specified in tc)
+  
+    combine_realization.sh
+      combine.swift
+        combine.chk.sh (app, specified in tc)
 
-3. run:
 
-  ./runall.combine_realizations.sh >& runall.out
+To process a set of realizations (for example, to combine them), perform these
+steps:
 
+1. Add Swift and a recent Sun Java to your path. For now we will use a Swift
+   and Java version maintained by Swift team member Jon Monette:
+
+   PATH=/home/jonmon/Library/Swift/swift-0.93/bin:/home/jonmon/Library/jdk1.7.0_03/bin:$PATH
+
+2. Create and cd to a working directory below which runall.sh can make per-run
+   directories:
+
+   mkdir $HOME/swiftruns
+   cd    $HOME/swiftruns
+
+3. Check out these HiRAM tools from svn:
+
+   mkdir $HOME/HiRAMTools
+   cd $HOME/HiRAMTools
+
+   svn co https://svn.ci.uchicago.edu/svn/vdl2/SwiftApps/ParVis/HiRAMTools
+
+4. Copy runall.config to your swiftruns dir and edit it to specify your input
+   and output directories.  It contains for example:
+
+
+   realtodo=real.todo                                        # File of names of realization subdirs to process
+   realdone=real.done                                        # File of names of realization subdirs completed
+                                                             # (this should be an empty file at start)
+
+   outdir=/intrepid-fs0/users/wilde/persistent/LZ/yearly     # Base dir for yearly output
+
+   # script=combine_realization.sh                           # Script to run for each realization
+   script=makeyearly_realization.sh                          # Script to run for each realization
+
+
+5. Place the list of realizations to process in a file named "real.todo" and
+   create and empty file "real.done". For example, to process 2 realizations,
+   Climo_001 and Climo_023):
+
+   cat >real.todo
+   Climo_001
+   Climo_023
+   ^D
+
+   >real.done   # create an empty real.done file
+
+6. run:
+
+   ./runall.combine_realizations.sh >& runall.out
+
+Then look for the most recently created run directory, and do:
+
+   tail -f swift.out
+
+
+OPEN ISSUES:
+
+- We see straggler scripts on Eureka nodes. Seems to be doing very slow IO on
+shared disk for unexplained reasons. Ticket is open with ALCF support, being
+investigated by Andrew Cherry.

Deleted: SwiftApps/ParVis/HiRAMTools/checkgroups.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/checkgroups.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/checkgroups.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,21 +0,0 @@
-#! /bin/bash
-
-# /intrepid-fs0/users/lzamboni/persistent/en1eo8Ic1/history/19500101/atmos_4xdaily.tile1.nc
-
-sed -e 's/^.*history.//' groups | sort >todo
-find /intrepid-fs0/users/wilde/persistent/en1eo8Ic1/combined/run016 -type f -print | fgrep .nc | sed -e 's/^.*run016.//' | sort >done
-
-echo
-echo diff: "<todo >done"
-echo
-diff todo done
-
-echo
-echo last 20 files produced were:
-echo
-ls -ltd /intrepid-fs0/users/wilde/persistent/en1eo8Ic1/combined/run016/*/* | head -20
-
-echo
-echo File lengths:
-echo
-ls -ltd /intrepid-fs0/users/wilde/persistent/en1eo8Ic1/combined/run016/*/* | awk '{print $5}' | sort | uniq -c

Deleted: SwiftApps/ParVis/HiRAMTools/combine.chk.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/combine.chk.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/combine.chk.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,33 +0,0 @@
-#! /bin/bash
-
-set -o errexit
-
-mkdir -p /scratch/$USER
-
-tmp=$(mktemp -d /scratch/$USER/combine_XXXXXX)
-
-echo mktemp error code: $?
-
-cd $tmp
-
-outdir=$1
-inpattern=$2
-date=$(basename $(dirname $inpattern))
-basepattern=$(basename $inpattern)
-
-infiles="$2*"
-for f in $infiles; do
-  dd if=$f of=$(basename $f) bs=4M
-done
-
-/home/ckerr/bin/fms/tools/x86_64/bin/mppnccombine.exe out.nc ${basepattern}*
-
-mkdir -p $outdir/$date
-dd if=out.nc of=$outdir/$date/$basepattern.trans bs=4M
-mv $outdir/$date/$basepattern.trans $outdir/$date/$basepattern
-
-cd ..
-rm -rf $tmp
-
-
-

Deleted: SwiftApps/ParVis/HiRAMTools/combine.orig.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/combine.orig.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/combine.orig.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,28 +0,0 @@
-#! /bin/bash
-
-set -x 
-
-mkdir -p /scratch/wilde
-tmp=$(mktemp -d /scratch/wilde/combine_XXXX)
-cd $tmp
-
-outdir=$1
-inpattern=$2
-date=$(basename $(dirname $inpattern))
-basepattern=$(basename $inpattern)
-
-infiles="$2*"
-for f in $infiles; do
-  dd if=$f of=$(basename $f) bs=4M
-done
-
-/home/ckerr/bin/fms/tools/x86_64/bin/mppnccombine.exe out.nc ${basepattern}*
-
-mkdir -p $outdir/$date
-dd if=out.nc of=$outdir/$date/$basepattern bs=4M
-
-cd ..
-rm -rf $tmp
-
-
-

Deleted: SwiftApps/ParVis/HiRAMTools/combine.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/combine.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/combine.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,29 +0,0 @@
-#! /bin/bash
-
-set -x 
-
-mkdir -p /scratch/wilde
-tmp=$(mktemp -d /scratch/wilde/combine_XXXX)
-cd $tmp
-
-outdir=$1
-inpattern=$2
-date=$(basename $(dirname $inpattern))
-basepattern=$(basename $inpattern)
-
-infiles="$2*"
-for f in $infiles; do
-  dd if=$f of=$(basename $f) bs=4M
-done
-
-/home/ckerr/bin/fms/tools/x86_64/bin/mppnccombine.exe out.nc ${basepattern}*
-
-mkdir -p $outdir/$date
-dd if=out.nc of=$outdir/$date/$basepattern.trans bs=4M
-mv $outdir/$date/$basepattern.trans $outdir/$date/$basepattern
-
-cd ..
-rm -rf $tmp
-
-
-

Added: SwiftApps/ParVis/HiRAMTools/combine.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/combine.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/combine.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,29 @@
+#! /bin/bash
+
+set -x 
+
+mkdir -p /scratch/wilde
+tmp=$(mktemp -d /scratch/wilde/combine_XXXX)
+cd $tmp
+
+outdir=$1
+inpattern=$2
+date=$(basename $(dirname $inpattern))
+basepattern=$(basename $inpattern)
+
+infiles="$2*"
+for f in $infiles; do
+  dd if=$f of=$(basename $f) bs=4M
+done
+
+/home/ckerr/bin/fms/tools/x86_64/bin/mppnccombine.exe out.nc ${basepattern}*
+
+mkdir -p $outdir/$date
+dd if=out.nc of=$outdir/$date/$basepattern.trans bs=4M
+mv $outdir/$date/$basepattern.trans $outdir/$date/$basepattern
+
+cd ..
+rm -rf $tmp
+
+
+

Deleted: SwiftApps/ParVis/HiRAMTools/combine_realization.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/combine_realization.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/combine_realization.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,82 +0,0 @@
-#! /bin/bash
-
-hdir=$1
-outdir=$2
-
-/bin/ls -1 -d ${hdir}* >dirs
-
-echo '=== ' $(basename $0) doing dirs:
-cat dirs
-
-for d in $(cat dirs); do
-  find $d | grep .nc.[0-9][0-9][0-9][0-9]  | sed -e 's/.....$//' | sort | uniq
-done >groups
-
-# Create new runNNN directory
-
-rundir=$( echo run??? | sed -e 's/^.*run//' | awk '{ printf("run%03d\n", $1+1)}' )
-mkdir $rundir
-
-cat >$rundir/cf <<END
-
-wrapperlog.always.transfer=true
-sitedir.keep=true
-execution.retries=10
-lazy.errors=true
-
-END
-
-cat >$rundir/sites.xml <<END
-
-<config>
-
-   <pool handle="local">
-     <execution provider="local" />
-     <profile namespace="karajan" key="jobThrottle">.01</profile>
-     <profile namespace="karajan" key="initialScore">10000</profile>
-     <filesystem provider="local"/>
-     <workdirectory>$PWD/swiftwork</workdirectory>
-   </pool>
-
-  <pool handle="cobalt">
-    <execution provider="coaster" jobmanager="local:cobalt" url="eureka.alcf.anl.gov" />
-    <filesystem provider="local" />
-    <!-- <profile namespace="globus"key="internalHostname">172.17.3.11</profile> -->
-    <!-- <profile namespace="globus"  key="project">MTCScienceApps</profile> -->
-    <profile namespace="globus"  key="queue">default</profile>
-    <profile namespace="karajan" key="jobthrottle">2.56</profile>
-    <profile namespace="karajan" key="initialScore">10000</profile>
-    <profile namespace="globus"  key="jobsPerNode">8</profile>
-    <!-- <profile namespace="globus"  key="workerLoggingLevel">DEBUG</profile> -->
-    <!-- <profile namespace="globus"  key="workerLoggingDirectory">/home/jonmon/Workspace/Swift/tests/catsnsleep</profile> -->
-    <profile namespace="globus"  key="slots">32</profile>
-    <profile namespace="globus"  key="maxTime">28000</profile> <!-- ~ 8hrs -->
-    <profile namespace="globus"  key="maxWallTime">00:20:00</profile>
-    <profile namespace="globus"  key="lowoverallocation">100</profile>
-    <profile namespace="globus"  key="highoverallocation">100</profile>
-    <profile namespace="globus"  key="nodeGranularity">1</profile>
-    <profile namespace="globus"  key="maxNodes">1</profile>
-    <workdirectory>$PWD/$rundir/swiftwork</workdirectory>
-  </pool>
-
-</config>
-
-END
-
-cat >$rundir/tc <<END
-
-#local  combine $PWD/combine.sh
-cobalt  combine $PWD/combine.chk.sh
-
-END
-
-cp groups combine.swift $rundir
-
-cd $rundir
-echo Running in directory $rundir
-runid=$(basename $rundir)
-mkdir -p $outdir/$runid
-
-swift -config cf -tc.file tc -sites.file sites.xml combine.swift -rundir=$outdir/$runid -groupfile=groups >& swift.out
-
-

Added: SwiftApps/ParVis/HiRAMTools/combine_realization.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/combine_realization.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/combine_realization.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,82 @@
+#! /bin/bash
+
+hdir=$1
+outdir=$2
+
+/bin/ls -1 -d ${hdir}* >dirs
+
+echo '=== ' $(basename $0) doing dirs:
+cat dirs
+
+for d in $(cat dirs); do
+  find $d | grep .nc.[0-9][0-9][0-9][0-9]  | sed -e 's/.....$//' | sort | uniq
+done >groups
+
+# Create new runNNN directory
+
+rundir=$( echo run??? | sed -e 's/^.*run//' | awk '{ printf("run%03d\n", $1+1)}' )
+mkdir $rundir
+
+cat >$rundir/cf <<END
+
+wrapperlog.always.transfer=true
+sitedir.keep=true
+execution.retries=10
+lazy.errors=true
+
+END
+
+cat >$rundir/sites.xml <<END
+
+<config>
+
+   <pool handle="local">
+     <execution provider="local" />
+     <profile namespace="karajan" key="jobThrottle">.01</profile>
+     <profile namespace="karajan" key="initialScore">10000</profile>
+     <filesystem provider="local"/>
+     <workdirectory>$PWD/swiftwork</workdirectory>
+   </pool>
+
+  <pool handle="cobalt">
+    <execution provider="coaster" jobmanager="local:cobalt" url="eureka.alcf.anl.gov" />
+    <filesystem provider="local" />
+    <!-- <profile namespace="globus"key="internalHostname">172.17.3.11</profile> -->
+    <!-- <profile namespace="globus"  key="project">MTCScienceApps</profile> -->
+    <profile namespace="globus"  key="queue">default</profile>
+    <profile namespace="karajan" key="jobthrottle">2.56</profile>
+    <profile namespace="karajan" key="initialScore">10000</profile>
+    <profile namespace="globus"  key="jobsPerNode">8</profile>
+    <!-- <profile namespace="globus"  key="workerLoggingLevel">DEBUG</profile> -->
+    <!-- <profile namespace="globus"  key="workerLoggingDirectory">/home/jonmon/Workspace/Swift/tests/catsnsleep</profile> -->
+    <profile namespace="globus"  key="slots">32</profile>
+    <profile namespace="globus"  key="maxTime">28000</profile> <!-- ~ 8hrs -->
+    <profile namespace="globus"  key="maxWallTime">00:20:00</profile>
+    <profile namespace="globus"  key="lowoverallocation">100</profile>
+    <profile namespace="globus"  key="highoverallocation">100</profile>
+    <profile namespace="globus"  key="nodeGranularity">1</profile>
+    <profile namespace="globus"  key="maxNodes">1</profile>
+    <workdirectory>$PWD/$rundir/swiftwork</workdirectory>
+  </pool>
+
+</config>
+
+END
+
+cat >$rundir/tc <<END
+
+#local  combine $PWD/combine.sh
+cobalt  combine $PWD/combine.chk.sh
+
+END
+
+cp groups combine.swift $rundir
+
+cd $rundir
+echo Running in directory $rundir
+runid=$(basename $rundir)
+mkdir -p $outdir/$runid
+
+swift -config cf -tc.file tc -sites.file sites.xml combine.swift -rundir=$outdir/$runid -groupfile=groups >& swift.out
+
+

Deleted: SwiftApps/ParVis/HiRAMTools/find_anomalies.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/find_anomalies.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/find_anomalies.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,18 +0,0 @@
-#! /bin/sh
-
-reals=/intrepid-fs0/users/wilde/persistent/LZ/combined
-cd $reals
-
-for real in *; do
-  echo Realization: $real
-  ( cd $real
-    for y in *; do
-      echo "  Year: " $y
-      ( cd $y
-        for f in *.nc; do
-          echo -n $f; ncdump -h $f | grep 'time =' || echo
-        done | awk '{print "    ",$1,$7}' | grep 'atmos.*\.tile[0-9].* (1' | grep -v scalar
-      )
-    done
-  )
-done

Deleted: SwiftApps/ParVis/HiRAMTools/get_atmos_scalar.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/get_atmos_scalar.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/get_atmos_scalar.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,17 +0,0 @@
-#! /bin/bash
-
-src=/intrepid-fs0/users/lzamboni/persistent
-dest=/intrepid-fs0/users/wilde/persistent/LZ/combined
-
-f=atmos_scalar.nc
-
-reals=$(cat /home/wilde/LZ/realizations)
-
-for r in $reals; do
-  cd $src/$r/history
-  years=$( /bin/ls -1d [0-9]??????? )
-  for y in $years; do
-    cp $y/$f $dest/$r/$y
-  done
-done
-

Deleted: SwiftApps/ParVis/HiRAMTools/get_real_copy.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/get_real_copy.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/get_real_copy.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,26 +0,0 @@
-#! /bin/bash
-
-# src=/intrepid-fs0/users/lzamboni/persistent
-
-src=/intrepid-fs0/users/wilde/persistent/LZ/combined
-dest=$PWD
-
-real=$1
-
-if [ _$1 = _ ]; then
-  echo realization to copy was not specified - exiting; exit 1
-fi
-
-if [ -e _$1 ]; then
-  echo $1 already exits - exiting; exit 1
-fi
-
-cd $src/$real
-dirs=$(find * -type d -print)
-
-for d in $dirs; do
-  mkdir -p $dest/$real/$d
-  for f in $d/*.nc; do
-    ln -s $PWD/$f $dest/$real/$f
-  done
-done

Deleted: SwiftApps/ParVis/HiRAMTools/get_real_copyL.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/get_real_copyL.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/get_real_copyL.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,26 +0,0 @@
-#! /bin/bash
-
-# src=/intrepid-fs0/users/lzamboni/persistent
-
-src=/intrepid-fs0/users/wilde/persistent/LZ/combined
-dest=$PWD
-
-real=$1
-
-if [ _$1 = _ ]; then
-  echo realization to copy was not specified - exiting; exit 1
-fi
-
-if [ -e _$1 ]; then
-  echo $1 already exits - exiting; exit 1
-fi
-
-cd $src/$real
-dirs=$(find * -type d -print)
-
-for d in $dirs; do
-  mkdir -p $dest/$real/history/$d
-  for f in $d/*.nc; do
-    ln -s $PWD/$f $dest/$real/history/$f
-  done
-done

Deleted: SwiftApps/ParVis/HiRAMTools/makeyearly-cdo.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/makeyearly-cdo.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/makeyearly-cdo.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,56 +0,0 @@
-#! /bin/bash
-
-# Create a fast (local-filesystem) working dir
-
-if mkdir -p /scratch/wilde; then
-  scratch=/scratch/wilde
-elif mkdir -p $HOME/scratch; then
-  scratch=$HOME/scratch
-else
-  echo $0: cant create scratch dir
-  exit 11
-fi
-
-tmp=$(mktemp -d $scratch/makeyearly_XXXXXX)
-startdir=$PWD
-cd $tmp
-
-echo running in $PWD
-
-qtrdec1=$1  # 1st quarter-decade dir
-qtrdec2=$2  # 2nd quarter-decade dir
-destbase=$3 # base dir for output years
-
-echo destbase=$destbase
-
-export PATH=/soft/apps/cdo-1.4.7/bin:$PATH
-export LD_LIBRARY_PATH=/soft/apps/intel-fc-11.0.074/lib/intel64:/soft/apps/intel-cc-11.0.074/lib/intel64:/soft/apps/netcdf-3.6.3/lib:/usr/lib:/usr/X11R6/lib:/usr/local/lib
-
-fname=$(basename $qtrdec1 .nc)
-firstyear=$(basename $(dirname $qtrdec1) | sed -e 's/....$//')
-
-if echo $fname | egrep -v 'grid_spec|scalar'; then
-  cdo mergetime $qtrdec1 $qtrdec2 ${fname}.nc
-  cdo splityear ${fname}.nc ${fname}.
-  # Yields, eg, from atmos_month.tile1:
-  # atmos_month.tile1.1950.nc
-  # atmos_month.tile1.1951.nc
-  # atmos_month.tile1.1952.nc
-  # atmos_month.tile1.1953.nc
-  # atmos_month.tile1.1954.nc
-fi  
-
-for (( y=0; y<5; y++ ))
-do
-  year=$((firstyear+y))
-  yeardir=${year}0101
-  destdir=$destbase/$yeardir
-  mkdir -p $destdir
-  if echo $fname | egrep -v 'grid_spec|scalar'; then
-    dd if=$fname.$year.nc of=$destdir/$fname.nc bs=8M
-  else
-    dd if=$qtrdec1 of=$destdir/$fname.nc bs=8M
-  fi
-done
-
-rm -rf $tmp

Added: SwiftApps/ParVis/HiRAMTools/makeyearly-cdo.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/makeyearly-cdo.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/makeyearly-cdo.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,82 @@
+#! /bin/bash
+
+# Create a fast (local-filesystem) working dir
+
+if mkdir -p /scratch/wilde; then
+  scratch=/scratch/wilde
+elif mkdir -p $HOME/scratch; then
+  scratch=$HOME/scratch
+else
+  echo $0: cant create scratch dir
+  exit 11
+fi
+
+tmp=$(mktemp -d $scratch/makeyearly_XXXXXX)
+startdir=$PWD
+cd $tmp
+
+echo running in $PWD
+
+qtrdec1=$1  # 1st quarter-decade dir
+qtrdec2=$2  # 2nd quarter-decade dir
+destbase=$3 # base dir for output years
+
+echo destbase=$destbase
+
+# Add cdo and nco tools to PATH and Intel compiler v11 libs to LD_LIBRARY_PATH
+
+export PATH=/soft/apps/cdo-1.4.7/bin:/soft/apps/nco-4.0.9/bin:$PATH
+export LD_LIBRARY_PATH=/soft/apps/intel-fc-11.0.074/lib/intel64:/soft/apps/intel-cc-11.0.074/lib/intel64:/soft/apps/netcdf-3.6.3/lib:/usr/lib:/usr/X11R6/lib:/usr/local/lib
+
+fname=$(basename $qtrdec1 .nc)
+firstyear=$(basename $(dirname $qtrdec1) | sed -e 's/....$//')
+
+# process only the time-based files
+
+if echo $fname | egrep -v 'grid_spec|scalar'; then
+
+  # Merge 2 quarter-decade files into one half-decade file
+
+  ncrcat $qtrdec1 $qtrdec2 5years.nc 
+
+  # Extract each year from the half-decade file
+
+  for (( y=$firstyear; y<$firstyear+5; y++ )); do
+    ncks -d time,"${y}-01-01 00:00:00","${y}-12-31 23:59:59" 5years.nc  $y.nc
+  done
+
+fi  
+
+# Copy the years from temporary to permanent filesystem
+
+for (( y=$firstyear; y<$firstyear+5; y++ ))
+do
+  destdir=$destbase/${y}0101
+  mkdir -p $destdir
+  if echo $fname | egrep -v 'grid_spec|scalar'; then
+    dd if=$y.nc of=$destdir/$fname.nc bs=8M
+  else
+    dd if=$qtrdec1 of=$destdir/$fname.nc bs=8M
+  fi
+done
+
+rm -rf $tmp
+
+# Delete when this works:
+
+  # Old method:
+  #
+  # cdo mergetime $qtrdec1 $qtrdec2 ${fname}.nc
+  # cdo splityear ${fname}.nc ${fname}.
+  # Yields, eg, from atmos_month.tile1:
+  # atmos_month.tile1.1950.nc
+  # atmos_month.tile1.1951.nc
+  # atmos_month.tile1.1952.nc
+  # atmos_month.tile1.1953.nc
+  # atmos_month.tile1.1954.nc
+
+  # New method:
+  #
+  # ncrcat input1.nc input2.nc 5years.nc 
+  # ncks -d time,"$yyyy-01-01 00:00:00","$yyyy-12-31 23:59:59" 5years.nc  year1.nc
+

Deleted: SwiftApps/ParVis/HiRAMTools/makeyearly.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/makeyearly.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/makeyearly.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,231 +0,0 @@
-#! /bin/bash
-
-set -x
-
-infile=$1  # must be full path
-outdir=$2  # must be full path
-
-realdir=$(dirname $(dirname $1))  # realization dir, e.g., en*
-realname=$(basename $realdir)      # realization name
-f=$(basename $1)
-d1=$(dirname $1)
-yfirst=$(echo $(basename $d1) | sed -e 's/....$//')
-ymid=$((yfirst+2))
-#ydir1=${yfirst}0101
-d2=$realdir/${ymid}0701
-
-period=$(basename $f .nc | sed -e 's/^.*_//' -e 's/\..*$//' -e 's/_.*//')
-
-leap()
-{
-  case $1 in
-    1952|1956|1960|1964|1968|1972|1976|1980|1984|1988|1992|1996|2000) echo 1 ;;
-    *) echo 0 ;;
-  esac
-}
-
-units()
-{
-  p=$1 # period
-  y=$2 # year
-  case $period in
-    4xdaily)   echo $((4*(365+$(leap $y)))) ;;
-    8xdaily)   echo $((8*(365+$(leap $y)))) ;;
-    daily)     echo $((1*(365+$(leap $y)))) ;;
-    month)     echo 12                     ;;
-    * )
-  esac
-}
-
-mkdir -p /scratch/wilde
-tmp=$(mktemp -d /scratch/wilde/makeyearly_XXXXXX)
-startdir=$PWD
-cd $tmp
-
-export PATH=$PATH:/soft/apps/nco-3.9.9/bin
-# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/soft/apps/nco-3.9.9/lib
-
-o=$outdir/$realname
-
-yd1=$o/$((yfirst+0))0101
-yd2=$o/$((yfirst+1))0101
-yd3=$o/$((yfirst+2))0101
-yd4=$o/$((yfirst+3))0101
-yd5=$o/$((yfirst+4))0101
-
-y1=$((yfirst+0))
-y2=$((yfirst+1))
-y3=$((yfirst+2))
-y4=$((yfirst+3))
-y5=$((yfirst+4))
-
-set +x
-e=echo
-
-$e mkdir -p $yd1 $yd2 $yd3 $yd4 $yd5
-
-$e dd if=$d1/$f of=$f bs=4M           # copy in first 2.5 year fle
-
-su=1                                  # start unit
-u1=$(units $period $y1)
-eu=$u1                                # end unit
-
-$e ncks -F -d time,$su,$eu $f out.nc   # extract year 1
-$e dd if=out.nc of=$yd1/$f bs=4M
-$e rm out.nc
-
-su=$((eu+1))
-u2=$(units $period $y2)
-eu=$((eu+$u2))
-
-$e ncks -F -d time,$su,$eu $f out.nc  # extract year 2
-$e dd if=out.nc of=$yd2/$f bs=4M
-$e rm out.nc
-
-su=$((eu+1))
-u3=$(units $period $y3)
-heu=$((u3/2))
-eu=$((eu+$heu))
-
-$e ncks -F -d time,$su,$eu $f tmp.a.nc # extract year 3 part a
-$e dd if=$d2/$f of=$f bs=4M            # copy in second 2.y year file
-
-su=1
-eu=$heu
-
-$e ncks -F -d time,$su,$eu $f tmp.b.nc # extract year 3 part b
-
-$e ncrcat tmp.a.nc tmp.b.nc out.nc     # merge year3 parts a and b
-$e dd if=out.nc of=$yd3/$f bs=4M
-$e rm out.nc tmp.a.nc tmp.b.nc
-
-su=$((eu+1))
-u4=$(units $period $y4)
-eu=$((eu+$u4))
-
-$e ncks -F -d time,$su,$eu $f out.nc   # extract year 4
-$e dd if=out.nc of=$yd4/$f bs=4M
-$e rm out.nc
-
-su=$((eu+1))
-u5=$(units $period $y5)
-eu=$((eu+$u4))
-
-$e ncks -F -d time,$su,$eu $f out.nc  # extract year 5
-$e dd if=out.nc of=$yd5/$f bs=4M
-$e rm out.nc $f
-
-$e cd $startdir
-rm -rf $tmp
-
-exit
-
-######################
-#
-
-the origin of the problem is that in the first 6 month of the year (Jan-Jun)
-there are less days than in the second half of the year (Jul-Dec), and that
-2.5 years refers to the # of months.
-
-The difference is 3 days in a normal year, 2 in a leap year.
-
-So the variations are not simply due by the presence of a leap year. These are
-the categories.  Let me define these before:
- 
-# normal= no leap in the 2.5 directory.
-# 1st half=directoryName is yyyy0101
-# 2nd half=directoryName is yyyy0701
-# 
-# N1 normal, 1st half -->3644 steps (911 days*4)
-# N2 normal, 2nd half -->3656 steps (914 days*4)
-# L1 leap, 1st half -->3648 steps   (912 days*4)
-# L2 leap, 2nd half -->3660 steps.  (915 days*4)
-
-# leap: 1952 1956 1960 1964 1968 1972 1976 1980 1984 1988 1992 1996 2000
-
-# Jan, Feb, Mar, Apr, May, June = (31+28+31+30+31+30) = 181
-
-# N-year: 181+184 x4= 724 + 736 = 1460
-# L-year: 182+184 x4= 728 + 736 = 1474
-
-
-19500101 1   L1: 3648 
-1951     1
-1952     / L
-
-19520701 /   N2: 3656
-1953     1
-1954     1
-----
-19550101 1   L1: 3648
-1956     1 L
-1957     /
-
-19570701 +   N2: 3656
-1958     1
-1959     1
-----
-19600101 1 L L1: 3648
-1961     1
-1962     /
-
-19620701 +   L2: 3660
-1963     1
-1964     1 L
-----
-19650101 1   N1: 3648
-1966     1
-1967     /
-
-19670701 +   L2: 3660
-1968     1 L
-1969     1
-----
-19700101 1   L1: 3648
-1971     1
-1972     / L
-
-19720701 +   N2: 3656
-1973     1
-1974     1
-----
-19750101 1   L1: 3648
-1976     1 L
-1977     /
-
-19770701 +   N2: 3656
-1978     1
-1979     1
-----
-19800101 1 L L1: 3648
-1981     1
-1982     /
-
-19820701 +   L2: 3660
-1983     1
-1984     1 L
-----
-19850101 1   N1: 3648
-1986     1
-1987     /
-
-19870701 +   L2: 3660
-1988     1 L
-1989     1
-----
-19900101 1   L1: 3648
-1991     1
-1992     / L
-
-19920701 +   N2: 3656
-1993     1
-1994     1
-----
-19950101 1   L1: 3648
-1995     1
-1996     / L
-
-19970701 +   N2: 3656
-1998     1
-1999     1
-----

Deleted: SwiftApps/ParVis/HiRAMTools/makeyearly_realization.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/makeyearly_realization.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/makeyearly_realization.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,96 +0,0 @@
-#! /bin/bash
-
-cdir=$1     # combined dir
-outbase=$2  # base of outdir for equivalent yearly files
-
-bindir=$(dirname $0)
-bindir=$(cd $bindir; pwd)
-
-echo bindir=$bindir
-
-# Enable one of the following patterns to name the output dir:
-
-outsuffix=$(basename $cdir)                              # rname
-outsuffix=$(basename $(dirname $cdir))                   # rname/rundir, use rname
-outsuffix=$(basename $(dirname $cdir))/$(basename $cdir) # rname/rundir, use both
-
-# generate the list of files to process: do every half-decade
-
-echo f1 f2 >ncfiles
-for y in $(seq 1950 5 1997); do
-  startyear=${y}0101; 
-  nextyear=$((y+2))0701
-  find $cdir/$startyear -type f | fgrep .nc | sed -e "s,^\\(.*\\)/$startyear/\\(.*\\),\\1/$startyear/\\2 \\1/$nextyear/\\2 $outdir,"
-done >>ncfiles
-
-# Create new runNNN directory
-
-rundir=$( echo run??? | sed -e 's/^.*run//' | awk '{ printf("run%03d\n", $1+1)}' )
-mkdir $rundir
-
-# Generate Swift config files (cf, tc, sites.xml):
-
-cat >$rundir/cf <<END
-
-wrapperlog.always.transfer=true
-sitedir.keep=true
-execution.retries=10
-lazy.errors=true
-status.mode=provider
-
-END
-
-cat >$rundir/sites.xml <<END
-
-<config>
-
-   <pool handle="local">
-     <execution provider="local" />
-     <profile namespace="karajan" key="jobThrottle">.01</profile>
-     <profile namespace="karajan" key="initialScore">10000</profile>
-     <filesystem provider="local"/>
-     <workdirectory>$PWD/swiftwork</workdirectory>
-   </pool>
-
-  <pool handle="cobalt">
-    <execution provider="coaster" jobmanager="local:cobalt" url="eureka.alcf.anl.gov" />
-    <filesystem provider="local" />
-    <!-- <profile namespace="globus"key="internalHostname">172.17.3.11</profile> -->
-    <profile namespace="globus"  key="project">MTCScienceApps</profile>
-    <profile namespace="globus"  key="queue">default</profile>
-    <profile namespace="karajan" key="jobthrottle">2.56</profile>
-    <profile namespace="karajan" key="initialScore">10000</profile>
-    <profile namespace="globus"  key="jobsPerNode">8</profile>
-    <!-- <profile namespace="globus"  key="workerLoggingLevel">DEBUG</profile> -->
-    <!-- <profile namespace="globus"  key="workerLoggingDirectory">/home/jonmon/Workspace/Swift/tests/catsnsleep</profile> -->
-    <profile namespace="globus"  key="slots">32</profile>
-    <profile namespace="globus"  key="maxTime">28000</profile> <!-- ~ 8hrs -->
-    <profile namespace="globus"  key="maxWallTime">00:20:00</profile>
-    <profile namespace="globus"  key="lowoverallocation">100</profile>
-    <profile namespace="globus"  key="highoverallocation">100</profile>
-    <profile namespace="globus"  key="nodeGranularity">1</profile>
-    <profile namespace="globus"  key="maxNodes">1</profile>
-    <workdirectory>$PWD/$rundir/swiftwork</workdirectory>
-  </pool>
-
-</config>
-
-END
-
-cat >$rundir/tc <<END
-
-cobalt makeyearly $bindir/makeyearly-cdo.sh
-
-END
-
-# Place file list and swift script in run dir, set params, make output dir, and run script
-
-cp ncfiles $bindir/makeyearly.swift $rundir
-
-cd $rundir
-echo Running in directory $rundir
-runid=$(basename $rundir) # un-needed?
-outdir=$outbase/$outsuffix/$runid
-mkdir -p $outdir
-
-swift -config cf -tc.file tc -sites.file sites.xml makeyearly.swift -rundir=$outdir -ncfiles=ncfiles >& swift.out

Added: SwiftApps/ParVis/HiRAMTools/makeyearly_realization.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/makeyearly_realization.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/makeyearly_realization.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,97 @@
+#! /bin/bash
+
+cdir=$1     # combined dir
+outbase=$2  # base of outdir for equivalent yearly files
+
+bindir=$(dirname $0)
+bindir=$(cd $bindir; pwd)
+
+echo bindir=$bindir
+
+# Enable one of the following patterns to name the output dir:
+
+outsuffix=$(basename $cdir)                              # rname
+outsuffix=$(basename $(dirname $cdir))                   # rname/rundir, use rname
+outsuffix=$(basename $(dirname $cdir))/$(basename $cdir) # rname/rundir, use both
+
+# generate the list of files to process: do every half-decade
+
+echo f1 f2 >ncfiles
+for y in $(seq 1950 5 1997); do
+  startyear=${y}0101; 
+  nextyear=$((y+2))0701
+  find $cdir/$startyear -type f | fgrep .nc | sed -e "s,^\\(.*\\)/$startyear/\\(.*\\),\\1/$startyear/\\2 \\1/$nextyear/\\2 $outdir,"
+done >>ncfiles
+
+# Create new runNNN directory
+
+rundir=$( echo run??? | sed -e 's/^.*run//' | awk '{ printf("run%03d\n", $1+1)}' )
+mkdir $rundir
+
+# Generate Swift config files (cf, tc, sites.xml):
+
+cat >$rundir/cf <<END
+
+wrapperlog.always.transfer=true
+sitedir.keep=true
+execution.retries=10
+lazy.errors=true
+status.mode=provider
+
+END
+
+cat >$rundir/sites.xml <<END
+
+<config>
+
+   <pool handle="local">
+     <execution provider="local" />
+     <profile namespace="karajan" key="jobThrottle">.01</profile>
+     <profile namespace="karajan" key="initialScore">10000</profile>
+     <filesystem provider="local"/>
+     <workdirectory>$PWD/swiftwork</workdirectory>
+   </pool>
+
+  <pool handle="cobalt">
+    <execution provider="coaster" jobmanager="local:cobalt" url="eureka.alcf.anl.gov" />
+    <filesystem provider="local" />
+    <!-- <profile namespace="globus"key="internalHostname">172.17.3.11</profile> -->
+    <!-- <profile namespace="globus"  key="project">MTCScienceApps</profile> -->
+    <profile namespace="globus"  key="project">SU_Climate</profile>
+    <profile namespace="globus"  key="queue">default</profile>
+    <profile namespace="karajan" key="jobthrottle">2.56</profile>
+    <profile namespace="karajan" key="initialScore">10000</profile>
+    <profile namespace="globus"  key="jobsPerNode">8</profile>
+    <!-- <profile namespace="globus"  key="workerLoggingLevel">DEBUG</profile> -->
+    <!-- <profile namespace="globus"  key="workerLoggingDirectory">/home/jonmon/Workspace/Swift/tests/catsnsleep</profile> -->
+    <profile namespace="globus"  key="slots">32</profile>
+    <profile namespace="globus"  key="maxTime">28000</profile> <!-- ~ 8hrs -->
+    <profile namespace="globus"  key="maxWallTime">00:20:00</profile>
+    <profile namespace="globus"  key="lowoverallocation">100</profile>
+    <profile namespace="globus"  key="highoverallocation">100</profile>
+    <profile namespace="globus"  key="nodeGranularity">1</profile>
+    <profile namespace="globus"  key="maxNodes">1</profile>
+    <workdirectory>$PWD/$rundir/swiftwork</workdirectory>
+  </pool>
+
+</config>
+
+END
+
+cat >$rundir/tc <<END
+
+cobalt makeyearly $bindir/makeyearly-cdo.sh
+
+END
+
+# Place file list and swift script in run dir, set params, make output dir, and run script
+
+cp ncfiles $bindir/makeyearly.swift $rundir
+
+cd $rundir
+echo Running in directory $rundir
+runid=$(basename $rundir) # un-needed?
+outdir=$outbase/$outsuffix/$runid
+mkdir -p $outdir
+
+swift -config cf -tc.file tc -sites.file sites.xml makeyearly.swift -rundir=$outdir -ncfiles=ncfiles >& swift.out

Deleted: SwiftApps/ParVis/HiRAMTools/run.combine_realization.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/run.combine_realization.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/run.combine_realization.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,5 +0,0 @@
-#! /bin/sh
-
-./combine_realization.sh  \
-  /intrepid-fs0/users/lzamboni/persistent/en1eo8Ic1/history/19 \
-  /intrepid-fs0/users/wilde/persistent/en1eo8Ic1/combined/ 

Deleted: SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.config
===================================================================
--- SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.config	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.config	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,4 +0,0 @@
-realdir=/intrepid-fs0/users/lzamboni/persistent           # Base dir of realizations
-realtodo=real.todo                                        # File of names of realization subdirs to process
-realdone=real.done                                        # File of names of realization subdirs completed
-outdir=/intrepid-fs0/users/wilde/persistent/LZ/combined4  # Base dir for combined output

Deleted: SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/runall.combine_realizations.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,50 +0,0 @@
-#! /bin/sh
-
-# ls -1d en*/history/[0-9][0-9][0-9][0-9]????   eur$ ls -1d en*/history/[0-9][0-9][0-9][0-9]????
-
-# The following can be made into parameters to this script. Set here for now.
-
-realdir=/intrepid-fs0/users/lzamboni/persistent          # Base dir of realizations
-realtodo=real.todo                                       # File of names of realization subdirs to process
-realdone=real.done                                       # File of names of realization subdirs completed
-outdir=/intrepid-fs0/users/wilde/persistent/LZ/combined  # Base dir for combined output
-
-source ./runall.combine_realizations.config              # can overwrite above config params
-
-# Helpful patterns for filename matching
-
-yyyy="[0-9][0-9][0-9][0-9]"
-yyyymmdd="${yyyy}[0-9][0-9][0-9][0-9]"
-
-mkdir -p $outdir
-
-while true; do
-
-  real=$(diff $realtodo $realdone | grep '^<' | sed -e 's/^..//' | head -1) # next realization to do
-  if [ _$real = _ ]; then
-    break
-  fi
-
-  echo Combining $real
-
-  find $realdir/$real/history/$yyyymmdd  |
-    grep .nc.$yyyy      |
-    sed -e 's/.....$//' | sort | uniq > $real.groups
-
-  echo Created $real.groups
-
-  echo  ./combine_realization.sh    \
-    "$realdir/$real/history/$yyyymmdd" \
-    $outdir/$real 
-
-  ./combine_realization.sh    \
-    "$realdir/$real/history/$yyyymmdd" \
-    $outdir/$real 
-
-  echo Completed $real
-  echo $real >>$realdone
-
-done
-
-echo All combines are done.
-

Modified: SwiftApps/ParVis/HiRAMTools/runall.config
===================================================================
--- SwiftApps/ParVis/HiRAMTools/runall.config	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/runall.config	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,6 +1,9 @@
+
 realtodo=real.todo                                        # File of names of realization subdirs to process
 realdone=real.done                                        # File of names of realization subdirs completed
+                                                          # (this should be an empty file at start)
+
 outdir=/intrepid-fs0/users/wilde/persistent/LZ/yearly     # Base dir for yearly output
 
-# script=combine_realizations.sh
-script=makeyearly_realization.sh
+# script=combine_realization.sh                           # Script to run for each realization
+script=makeyearly_realization.sh                          # Script to run for each realization

Deleted: SwiftApps/ParVis/HiRAMTools/runall.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/runall.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/runall.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,54 +0,0 @@
-#! /bin/sh
-
-bindir=$(dirname $0)
-bindir=$(cd $bindir; pwd)
-
-echo bindir=$bindir
-
-realtodo=real.todo                                       # File of full path names of realization dirs to process
-realdone=real.done                                       # File of full path names of realization dirs completed
-outdir=/intrepid-fs0/users/wilde/persistent/LZ/yearly      # Base dir for processed output
-
-source $bindir/runall.config # set default params
-if [ -f runall.config ]; then
-  source runall.config       # set local params
-fi
-
-echo bindir=$bindir script=$script
-
-# Helpful patterns for filename matching
-
-yyyy="[0-9][0-9][0-9][0-9]"
-yyyymmdd="${yyyy}[0-9][0-9][0-9][0-9]"
-
-mkdir -p $outdir
-
-while true; do
-
-  real=$(diff $realtodo $realdone | grep '^<' | sed -e 's/^..//' | head -1) # next realization to do
-  if [ _$real = _ ]; then
-    break
-  fi
-
-  echo RunAll: Processing $real
-
-# find $realdir/$real/history/$yyyymmdd  |
-#   grep .nc.$yyyy      |
-#   sed -e 's/.....$//' | sort | uniq > $real.groups
-#
-#  echo Created $real.groups
-
-  echo  Running: $bindir/$script    \
-    $real \
-    $outdir 
-
-  $bindir/$script    \
-    $real \
-    $outdir 
-
-  echo RunAll: Completed $real
-  echo $real >>$realdone
-
-done
-
-echo RunAll: all realizations are done.

Added: SwiftApps/ParVis/HiRAMTools/runall.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/runall.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/runall.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -0,0 +1,54 @@
+#! /bin/sh
+
+bindir=$(dirname $0)
+bindir=$(cd $bindir; pwd)
+
+echo bindir=$bindir
+
+realtodo=real.todo                                       # File of full path names of realization dirs to process
+realdone=real.done                                       # File of full path names of realization dirs completed
+outdir=/intrepid-fs0/users/wilde/persistent/LZ/yearly      # Base dir for processed output
+
+source $bindir/runall.config   # set default params
+if [ -f ./runall.config ]; then
+  source ./runall.config       # set local params
+fi
+
+echo bindir=$bindir script=$script
+
+# Helpful patterns for filename matching
+
+yyyy="[0-9][0-9][0-9][0-9]"
+yyyymmdd="${yyyy}[0-9][0-9][0-9][0-9]"
+
+mkdir -p $outdir
+
+while true; do
+
+  real=$(diff $realtodo $realdone | grep '^<' | sed -e 's/^..//' | head -1) # next realization to do
+  if [ _$real = _ ]; then
+    break
+  fi
+
+  echo RunAll: Processing $real
+
+# find $realdir/$real/history/$yyyymmdd  |
+#   grep .nc.$yyyy      |
+#   sed -e 's/.....$//' | sort | uniq > $real.groups
+#
+#  echo Created $real.groups
+
+  echo  Running: $bindir/$script    \
+    $real \
+    $outdir 
+
+  $bindir/$script    \
+    $real \
+    $outdir 
+
+  echo RunAll: Completed $real
+  echo $real >>$realdone
+
+done
+
+echo RunAll: all realizations are done.

Deleted: SwiftApps/ParVis/HiRAMTools/runall4.combine_realizations.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/runall4.combine_realizations.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/runall4.combine_realizations.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,48 +0,0 @@
-#! /bin/sh
-
-# ls -1d en*/history/[0-9][0-9][0-9][0-9]????   eur$ ls -1d en*/history/[0-9][0-9][0-9][0-9]????
-
-# The following can be made into parameters to this script. Set here for now.
-
-realdir=/intrepid-fs0/users/lzamboni/persistent           # Base dir of realizations
-realtodo=real4.todo                                       # File of names of realization subdirs to process
-realdone=real4.done                                       # File of names of realization subdirs completed
-outdir=/intrepid-fs0/users/wilde/persistent/LZ/combined4  # Base dir for combined output
-
-# Helpful patterns for filename matching
-
-yyyy="[0-9][0-9][0-9][0-9]"
-yyyymmdd="${yyyy}[0-9][0-9][0-9][0-9]"
-
-mkdir -p $outdir
-
-while true; do
-
-  real=$(diff $realtodo $realdone | grep '^<' | sed -e 's/^..//' | head -1) # next realization to do
-  if [ _$real = _ ]; then
-    break
-  fi
-
-  echo Combining $real
-
-  find $realdir/$real/history/$yyyymmdd  |
-    grep .nc.$yyyy      |
-    sed -e 's/.....$//' | sort | uniq > $real.groups
-
-  echo Created $real.groups
-
-  echo  ./combine_realization.sh    \
-    "$realdir/$real/history/$yyyymmdd" \
-    $outdir/$real 
-
-  ./combine_realization.sh    \
-    "$realdir/$real/history/$yyyymmdd" \
-    $outdir/$real 
-
-  echo Completed $real
-  echo $real >>$realdone
-
-done
-
-echo All combines are done.
-

Deleted: SwiftApps/ParVis/HiRAMTools/runcombine.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/runcombine.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/runcombine.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,5 +0,0 @@
-#! /bin/sh
-
-$HOME/LZ/combine.csh \
-    -a /intrepid-fs0/users/lzamboni/persistent/en1eo8Ic1/history \
-    -s 19500101 -n 0 -m 30 # -q

Deleted: SwiftApps/ParVis/HiRAMTools/y1.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/y1.sh	2012-06-05 16:45:58 UTC (rev 5803)
+++ SwiftApps/ParVis/HiRAMTools/y1.sh	2012-06-08 18:10:40 UTC (rev 5804)
@@ -1,29 +0,0 @@
-#! /bin/sh
-
-export PATH=$PATH:/soft/apps/nco-3.9.9/bin
-# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/soft/apps/nco-3.9.9/lib
-
-d1=19500101
-d2=19520701
-f=ice_month.nc
-
-o=testyears
-y1=$o/19500101
-y2=$o/19510101
-y3=$o/19520101
-y4=$o/19530101
-y5=$o/19540101
-
-mkdir -p $y1 $y2 $y3 $y4 $y5
-
-ncks -F -d time,1,12    $d1/$f $y1/$f
-
-ncks -F -d time,13,24   $d1/$f $y2/$f
-
-ncks -F -d time,25,30   $d1/$f $y3/$f.a
-ncks -F -d time,1,6     $d2/$f $y3/$f.b
-ncrcat $y3/$f.a $y3/$f.b       $y3/$f
-
-ncks -F -d time,7,18    $d2/$f $y4/$f
-
-ncks -F -d time,19,30   $d2/$f $y5/$f



From wilde at ci.uchicago.edu  Fri Jun  8 13:11:35 2012
From: wilde at ci.uchicago.edu (wilde at ci.uchicago.edu)
Date: Fri,  8 Jun 2012 13:11:35 -0500 (CDT)
Subject: [Swift-commit] r5805 - SwiftApps/ParVis/HiRAMTools
Message-ID: <20120608181135.1A9139CC99@svn.ci.uchicago.edu>

Author: wilde
Date: 2012-06-08 13:11:34 -0500 (Fri, 08 Jun 2012)
New Revision: 5805

Removed:
   SwiftApps/ParVis/HiRAMTools/combine2.swift
Log:
Remove junk file.

Deleted: SwiftApps/ParVis/HiRAMTools/combine2.swift
===================================================================
--- SwiftApps/ParVis/HiRAMTools/combine2.swift	2012-06-08 18:10:40 UTC (rev 5804)
+++ SwiftApps/ParVis/HiRAMTools/combine2.swift	2012-06-08 18:11:34 UTC (rev 5805)
@@ -1,18 +0,0 @@
-
-type file;
-
-string rundir=@arg("rundir","NO-OUTDIR-SPECIFIED");
-string groupfile=@arg("groupfile","groups");
-string realizatoinfile=@arg("ralizationsfile","realizations");
-
-app combine (string dir, string pattern)
-{
-  combine dir pattern;
-}
-
-string group[] = readData(groupfile);
-string reals[] = readData(realizationFile);
-
-foreach g in group {
-  combine(rundir,g);
-}



From wilde at ci.uchicago.edu  Fri Jun  8 13:54:46 2012
From: wilde at ci.uchicago.edu (wilde at ci.uchicago.edu)
Date: Fri,  8 Jun 2012 13:54:46 -0500 (CDT)
Subject: [Swift-commit] r5806 - in SwiftApps/ParVis/HiRAMTools: . JUNK
Message-ID: <20120608185446.680B79CC99@svn.ci.uchicago.edu>

Author: wilde
Date: 2012-06-08 13:54:46 -0500 (Fri, 08 Jun 2012)
New Revision: 5806

Added:
   SwiftApps/ParVis/HiRAMTools/get_real_copy.sh
   SwiftApps/ParVis/HiRAMTools/get_real_copyL.sh
Removed:
   SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copy.sh
   SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copyL.sh
Log:
Move util scripts back from JUNK.

Deleted: SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copy.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copy.sh	2012-06-08 18:11:34 UTC (rev 5805)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copy.sh	2012-06-08 18:54:46 UTC (rev 5806)
@@ -1,26 +0,0 @@
-#! /bin/bash
-
-# src=/intrepid-fs0/users/lzamboni/persistent
-
-src=/intrepid-fs0/users/wilde/persistent/LZ/combined
-dest=$PWD
-
-real=$1
-
-if [ _$1 = _ ]; then
-  echo realization to copy was not specified - exiting; exit 1
-fi
-
-if [ -e _$1 ]; then
-  echo $1 already exits - exiting; exit 1
-fi
-
-cd $src/$real
-dirs=$(find * -type d -print)
-
-for d in $dirs; do
-  mkdir -p $dest/$real/$d
-  for f in $d/*.nc; do
-    ln -s $PWD/$f $dest/$real/$f
-  done
-done

Deleted: SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copyL.sh
===================================================================
--- SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copyL.sh	2012-06-08 18:11:34 UTC (rev 5805)
+++ SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copyL.sh	2012-06-08 18:54:46 UTC (rev 5806)
@@ -1,26 +0,0 @@
-#! /bin/bash
-
-# src=/intrepid-fs0/users/lzamboni/persistent
-
-src=/intrepid-fs0/users/wilde/persistent/LZ/combined
-dest=$PWD
-
-real=$1
-
-if [ _$1 = _ ]; then
-  echo realization to copy was not specified - exiting; exit 1
-fi
-
-if [ -e _$1 ]; then
-  echo $1 already exits - exiting; exit 1
-fi
-
-cd $src/$real
-dirs=$(find * -type d -print)
-
-for d in $dirs; do
-  mkdir -p $dest/$real/history/$d
-  for f in $d/*.nc; do
-    ln -s $PWD/$f $dest/$real/history/$f
-  done
-done

Copied: SwiftApps/ParVis/HiRAMTools/get_real_copy.sh (from rev 5804, SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copy.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/get_real_copy.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/get_real_copy.sh	2012-06-08 18:54:46 UTC (rev 5806)
@@ -0,0 +1,26 @@
+#! /bin/bash
+
+# src=/intrepid-fs0/users/lzamboni/persistent
+
+src=/intrepid-fs0/users/wilde/persistent/LZ/combined
+dest=$PWD
+
+real=$1
+
+if [ _$1 = _ ]; then
+  echo realization to copy was not specified - exiting; exit 1
+fi
+
+if [ -e _$1 ]; then
+  echo $1 already exits - exiting; exit 1
+fi
+
+cd $src/$real
+dirs=$(find * -type d -print)
+
+for d in $dirs; do
+  mkdir -p $dest/$real/$d
+  for f in $d/*.nc; do
+    ln -s $PWD/$f $dest/$real/$f
+  done
+done

Copied: SwiftApps/ParVis/HiRAMTools/get_real_copyL.sh (from rev 5804, SwiftApps/ParVis/HiRAMTools/JUNK/get_real_copyL.sh)
===================================================================
--- SwiftApps/ParVis/HiRAMTools/get_real_copyL.sh	                        (rev 0)
+++ SwiftApps/ParVis/HiRAMTools/get_real_copyL.sh	2012-06-08 18:54:46 UTC (rev 5806)
@@ -0,0 +1,26 @@
+#! /bin/bash
+
+# src=/intrepid-fs0/users/lzamboni/persistent
+
+src=/intrepid-fs0/users/wilde/persistent/LZ/combined
+dest=$PWD
+
+real=$1
+
+if [ _$1 = _ ]; then
+  echo realization to copy was not specified - exiting; exit 1
+fi
+
+if [ -e _$1 ]; then
+  echo $1 already exits - exiting; exit 1
+fi
+
+cd $src/$real
+dirs=$(find * -type d -print)
+
+for d in $dirs; do
+  mkdir -p $dest/$real/history/$d
+  for f in $d/*.nc; do
+    ln -s $PWD/$f $dest/$real/history/$f
+  done
+done



From davidk at ci.uchicago.edu  Fri Jun  8 16:13:21 2012
From: davidk at ci.uchicago.edu (davidk at ci.uchicago.edu)
Date: Fri,  8 Jun 2012 16:13:21 -0500 (CDT)
Subject: [Swift-commit] r5807 - trunk/etc
Message-ID: <20120608211321.70D749CC99@svn.ci.uchicago.edu>

Author: davidk
Date: 2012-06-08 16:13:21 -0500 (Fri, 08 Jun 2012)
New Revision: 5807

Modified:
   trunk/etc/swift.properties
Log:
Disable wrapper staging by default


Modified: trunk/etc/swift.properties
===================================================================
--- trunk/etc/swift.properties	2012-06-08 18:54:46 UTC (rev 5806)
+++ trunk/etc/swift.properties	2012-06-08 21:13:21 UTC (rev 5807)
@@ -356,7 +356,7 @@
 # With wrapper staging, files are staged in and out by the 
 # swift wrapper, on the worker node.
 
-use.wrapper.staging=true
+use.wrapper.staging=false
 
 # The URL prefix used to access local files in wrapper
 # staging mode. Full URLs are created by concatenating:
@@ -364,4 +364,4 @@
 #	* the current swift work directory (as an absolute path)
 #	* the file in question (relative to the current work directory)
 #
-wrapper.staging.local.server=file://
+#wrapper.staging.local.server=file://



From davidk at ci.uchicago.edu  Fri Jun  8 17:00:59 2012
From: davidk at ci.uchicago.edu (davidk at ci.uchicago.edu)
Date: Fri,  8 Jun 2012 17:00:59 -0500 (CDT)
Subject: [Swift-commit] r5808 - trunk/tests/documentation/tutorial
Message-ID: <20120608220059.C7FA79CC99@svn.ci.uchicago.edu>

Author: davidk
Date: 2012-06-08 17:00:59 -0500 (Fri, 08 Jun 2012)
New Revision: 5808

Removed:
   trunk/tests/documentation/tutorial/sequential_iteration.foldout0000.expected
   trunk/tests/documentation/tutorial/sequential_iteration.foldout0001.expected
   trunk/tests/documentation/tutorial/sequential_iteration.foldout0002.expected
   trunk/tests/documentation/tutorial/sequential_iteration.foldout0003.expected
   trunk/tests/documentation/tutorial/swift.properties
Modified:
   trunk/tests/documentation/tutorial/sequential_iteration.clean.sh
   trunk/tests/documentation/tutorial/sequential_iteration.swift
Log:
use the updated sequential iteration script from the tutorial


Modified: trunk/tests/documentation/tutorial/sequential_iteration.clean.sh
===================================================================
--- trunk/tests/documentation/tutorial/sequential_iteration.clean.sh	2012-06-08 21:13:21 UTC (rev 5807)
+++ trunk/tests/documentation/tutorial/sequential_iteration.clean.sh	2012-06-08 22:00:59 UTC (rev 5808)
@@ -1,4 +1,4 @@
 #!/bin/bash
 set -x
-rm -rf sequential_iteration-* sequential_iteration.kml sequential_iteration.xml foldout* || exit 1
+rm -rf sequential_iteration-* sequential_iteration.kml sequential_iteration.xml || exit 1
 exit 0

Deleted: trunk/tests/documentation/tutorial/sequential_iteration.foldout0000.expected
===================================================================
--- trunk/tests/documentation/tutorial/sequential_iteration.foldout0000.expected	2012-06-08 21:13:21 UTC (rev 5807)
+++ trunk/tests/documentation/tutorial/sequential_iteration.foldout0000.expected	2012-06-08 22:00:59 UTC (rev 5808)
@@ -1 +0,0 @@
-793578934574893

Deleted: trunk/tests/documentation/tutorial/sequential_iteration.foldout0001.expected
===================================================================
--- trunk/tests/documentation/tutorial/sequential_iteration.foldout0001.expected	2012-06-08 21:13:21 UTC (rev 5807)
+++ trunk/tests/documentation/tutorial/sequential_iteration.foldout0001.expected	2012-06-08 22:00:59 UTC (rev 5808)
@@ -1 +0,0 @@
-16
\ No newline at end of file

Deleted: trunk/tests/documentation/tutorial/sequential_iteration.foldout0002.expected
===================================================================
--- trunk/tests/documentation/tutorial/sequential_iteration.foldout0002.expected	2012-06-08 21:13:21 UTC (rev 5807)
+++ trunk/tests/documentation/tutorial/sequential_iteration.foldout0002.expected	2012-06-08 22:00:59 UTC (rev 5808)
@@ -1 +0,0 @@
-2
\ No newline at end of file

Deleted: trunk/tests/documentation/tutorial/sequential_iteration.foldout0003.expected
===================================================================
--- trunk/tests/documentation/tutorial/sequential_iteration.foldout0003.expected	2012-06-08 21:13:21 UTC (rev 5807)
+++ trunk/tests/documentation/tutorial/sequential_iteration.foldout0003.expected	2012-06-08 22:00:59 UTC (rev 5808)
@@ -1 +0,0 @@
-1
\ No newline at end of file

Modified: trunk/tests/documentation/tutorial/sequential_iteration.swift
===================================================================
--- trunk/tests/documentation/tutorial/sequential_iteration.swift	2012-06-08 21:13:21 UTC (rev 5807)
+++ trunk/tests/documentation/tutorial/sequential_iteration.swift	2012-06-08 22:00:59 UTC (rev 5808)
@@ -1,18 +1,12 @@
-type counterfile;  
-  
-app (counterfile t) echo(string m) {   
-    echo m stdout=@filename(t);  
-}  
-  
-app (counterfile t) countstep(counterfile i) {  
-    wcl @filename(i) @filename(t);  
-}  
-  
-counterfile a[]  <simple_mapper;prefix="sequential_iteration.foldout">;  
-  
-a[0] = echo("793578934574893");  
-  
-iterate v {  
-  a[v+1] = countstep(a[v]);  
- trace("extract int value ", at extractint(a[v+1]));  
-} until (@extractint(a[v+1]) <= 1);  
+
+string alphabet[];
+alphabet[0] = "a";
+alphabet[1] = "b";
+alphabet[2] = "c";
+alphabet[3] = "d";
+alphabet[4] = "e";
+
+iterate i  
+{
+   tracef("Letter %i is: %s\n", i, alphabet[i]);
+}  until(i == 5);

Deleted: trunk/tests/documentation/tutorial/swift.properties
===================================================================
--- trunk/tests/documentation/tutorial/swift.properties	2012-06-08 21:13:21 UTC (rev 5807)
+++ trunk/tests/documentation/tutorial/swift.properties	2012-06-08 22:00:59 UTC (rev 5808)
@@ -1,7 +0,0 @@
-wrapperlog.always.transfer=true
-sitedir.keep=true
-execution.retries=0
-lazy.errors=false
-status.mode=provider
-use.provider.staging=false
-provider.staging.pin.swiftfiles=false



From davidk at ci.uchicago.edu  Sun Jun 10 00:10:52 2012
From: davidk at ci.uchicago.edu (davidk at ci.uchicago.edu)
Date: Sun, 10 Jun 2012 00:10:52 -0500 (CDT)
Subject: [Swift-commit] r5809 - trunk/bin
Message-ID: <20120610051052.8672A9CCFE@svn.ci.uchicago.edu>

Author: davidk
Date: 2012-06-10 00:10:52 -0500 (Sun, 10 Jun 2012)
New Revision: 5809

Modified:
   trunk/bin/start-coaster-service
Log:
Start OSG proxy via start-coaster-service


Modified: trunk/bin/start-coaster-service
===================================================================
--- trunk/bin/start-coaster-service	2012-06-08 22:00:59 UTC (rev 5808)
+++ trunk/bin/start-coaster-service	2012-06-10 05:10:52 UTC (rev 5809)
@@ -315,6 +315,7 @@
    echo Starting workers   
    if [ -n "$WORKER_RELAY_HOST" ]; then
       ssh $WORKER_USERNAME@$WORKER_RELAY_HOST "mkdir -p condor"
+      ssh $WORKER_USERNAME@$WORKER_RELAY_HOST "grid-proxy-init"
       ssh $WORKER_USERNAME@$WORKER_RELAY_HOST "WORKER_INIT_CMD='$WORKER_INIT_CMD' WORKER_LOGGING_LEVEL=$WORKER_LOGGING_LEVEL run-gwms-workers $EXECUTION_URL $CONDOR_WORKERS" 
    else
       mkdir -p condor



From davidk at ci.uchicago.edu  Mon Jun 11 12:43:58 2012
From: davidk at ci.uchicago.edu (davidk at ci.uchicago.edu)
Date: Mon, 11 Jun 2012 12:43:58 -0500 (CDT)
Subject: [Swift-commit] r5810 - trunk/tests/functions
Message-ID: <20120611174358.E9C869CCE5@svn.ci.uchicago.edu>

Author: davidk
Date: 2012-06-11 12:43:58 -0500 (Mon, 11 Jun 2012)
New Revision: 5810

Removed:
   trunk/tests/functions/swift.properties
Log:
use default swift.properties for swift function tests


Deleted: trunk/tests/functions/swift.properties
===================================================================
--- trunk/tests/functions/swift.properties	2012-06-10 05:10:52 UTC (rev 5809)
+++ trunk/tests/functions/swift.properties	2012-06-11 17:43:58 UTC (rev 5810)
@@ -1,353 +0,0 @@
-sites.file=${swift.home}/etc/sites.xml
-tc.file=${swift.home}/etc/tc.data
-
-#
-# The host name of the submit machine is used by GRAM as a callback
-# address to report the status of submitted jobs. In general, Swift
-# can automatically detect the host name of the local machine.
-# However, if the machine host name is improperly configured or if
-# it does not represent a valid DNS entry, certain services (such as
-# GRAM) will not be able to send job status notifications back to
-# the client. The value of this property can be an IP address.
-#
-# Format:
-#    hostname=string
-#
-
-
-#hostname=localhost
-
-#
-# A TCP port range can be specified to restrict the ports on which GRAM
-# callback services are started. This is likely needed if your submit
-# host is behind a firewall, in which case the firewall should be
-# configured to allow incoming connections on ports in the range.
-#
-# Format:
-#     tcp.port.range=start,end
-#
-
-#tcp.port.range=50000,50100
-
-#
-# false	- means an error will be immediately reported and cause the
-# 		workflow to abort. At this time remote jobs that are already
-#		running will not be canceled
-# true	- means that Swift will try to do as much work as possible and
-#		report all errors encountered at the end. However, "errors"
-#		here only applies to job execution errors. Certain errors
-#		that are related to the Swift implementation (should such
-#		errors occur) will still be reported eagerly.
-#
-# Default: false
-#
-lazy.errors=false
-
-#
-# What algorithm to use for caching of remote files. LRU (as in what
-# files to purge) is the only implementation right now. One can set
-# a target size (in bytes) for a host by using the swift:storagesize
-# profile for a host in sites.xml
-#
-# Default: LRU
-#
-caching.algorithm=LRU
-
-#
-# true       - generate a provenance graph in .dot format (Swift will
-#			 choose a random file name)
-# false      - do not generate a provenance graph
-# <filename> - generate a provenange graph in the give file name
-#
-# Default: false
-#
-pgraph=false
-
-
-#
-# graph properties for the provenance graph (.dot specific)
-#
-# Default: splines="compound", rankdir="TB"
-#
-pgraph.graph.options=splines="compound", rankdir="TB"
-
-
-#
-# node properties for the provenance graph (.dot specific)
-#
-# Default: color="seagreen", style="filled"
-#
-pgraph.node.options=color="seagreen", style="filled"
-
-#
-# true	- clustering of small jobs is enabled. Clustering works in the
-#       following way: If a job is clusterable (meaning that it has the
-#       GLOBUS::maxwalltime profile specified in tc.data and its value
-#       is less than the value of the "clustering.min.time" property) it will
-#       be put in a clustering queue. The queue is processed at intervals
-#       specified by the "clustering.queue.delay" property. The processing
-#       of the clustering queue consists of selecting compatible jobs and
-#		grouping them in clusters whose max wall time does not exceed twice
-#       the value of the "clustering.min.time" property. Two or more jobs are
-#       considered compatible if they share the same site and do not have
-#       conflicting profiles (e.g. different values for the same environment
-#       variable).
-# false	- clustering of small jobs is disabled.
-#
-# Default: false
-#
-clustering.enabled=false
-
-
-#
-# <seconds>	- the intervals at which the clustering queue is processed
-#
-# Default: 4
-#
-clustering.queue.delay=4
-
-#
-# <seconds>	- the threshold time for clustering
-#
-# Default: 60
-#
-clustering.min.time=60
-
-#
-# Kickstart is a useful tool that can be used to gather various information
-# about a remote process. Before it can be used it must be installed on the
-# remote site and the corresponding entry be set in the sites file.
-# This option allows controlling of how Swift uses Kickstart. The following
-# values are possible:
-# false - do not use Kickstart
-# true  - use Kickstart. If a job is scheduled on a site that does not have
-#       Kickstart installed, that job will fail.
-# maybe - Use Kickstart if installed (i.e. the entry is present in the sites
-#       file)
-#
-# Default: maybe
-#
-
-kickstart.enabled=maybe
-
-#
-# Indicates when Kickstart records should be fetched from the remote site:
-# true	- always transfer Kickstart records if Kickstart was used (see
-#		kickstart.enabled)
-# false	- only transfer Kickstart records if the job fails
-#
-# Default: false
-#
-
-kickstart.always.transfer=false
-
-#
-# Indicates when wrapper logs should be fetched from the remote site:
-# true	- always transfer wrapper logs
-# false	- only transfer wrapper logs if the job fails
-#
-# Default: false
-#
-
-wrapperlog.always.transfer=false
-
-###########################################################################
-#                          Throttling options                             #
-###########################################################################
-#
-# For the throttling parameters, valid values are either a positive integer
-# or "off" (without the quotes).
-#
-
-#
-# Limits the number of concurrent submissions for a workflow instance. This
-# throttle only limits the number of concurrent tasks (jobs) that are being
-# sent to sites, not the total number of concurrent jobs that can be run.
-# The submission stage in GRAM is one of the most CPU expensive stages (due
-# mostly to the mutual authentication and delegation). Having too many
-# concurrent submissions can overload either or both the submit host CPU
-# and the remote host/head node causing degraded performance.
-#
-# Default: 4
-#
-
-throttle.submit=4
-#throttle.submit=off
-
-#
-# Limits the number of concurrent submissions for any of the sites Swift will
-# try to send jobs to. In other words it guarantees that no more than the
-# value of this throttle jobs sent to any site will be concurrently in a state
-# of being submitted.
-#
-# Default: 2
-#
-
-throttle.host.submit=100
-#throttle.host.submit=off
-
-#
-# The Swift scheduler has the ability to limit the number of concurrent jobs
-# allowed on a site based on the performance history of that site. Each site
-# is assigned a score (initially 1), which can increase or decrease based
-# on whether the site yields successful or faulty job runs. The score for a
-# site can take values in the (0.1, 100) interval. The number of allowed jobs
-# is calculated using the following formula:
-# 	2 + score*throttle.score.job.factor
-# This means a site will always be allowed at least two concurrent jobs and
-# at most 2 + 100*throttle.score.job.factor. With a default of 4 this means
-# at least 2 jobs and at most 402.
-#
-# Default: 4
-#
-
-throttle.score.job.factor=4
-#throttle.score.job.factor=off
-
-
-#
-# Limits the total number of concurrent file transfers that can happen at any
-# given time. File transfers consume bandwidth. Too many concurrent transfers
-# can cause the network to be overloaded preventing various other signalling
-# traffic from flowing properly.
-#
-# Default: 4
-#
-
-throttle.transfers=4
-#throttle.transfers=off
-
-# Limits the total number of concurrent file operations that can happen at any
-# given time. File operations (like transfers) require an exclusive connection
-# to a site. These connections can be expensive to establish. A large number
-# of concurrent file operations may cause Swift to attempt to establish many
-# such expensive connections to various sites. Limiting the number of concurrent
-# file operations causes Swift to use a small number of cached connections and
-# achieve better overall performance.
-#
-# Default: 8
-#
-
-throttle.file.operations=8
-#throttle.file.operations=off
-
-# Indicates whether the working directory on the remote site should be
-# left intact even when the workflow completes successfully. This can be
-# used to inspect the site working directory for debugging purposes.
-#
-# Default: false
-#
-
-sitedir.keep=false
-
-# number of time a job will be retried if it fails (giving a maximum of
-# 1 + execution.retries attempts at execution)
-#
-
-execution.retries=2
-
-
-# Enables/disables replication. Replication is used to deal with jobs sitting
-# in batch queues for abnormally large amounts of time. If replication is enabled
-# and certain conditions are met, Swift creates and submits replicas of jobs, and
-# allows multiple instances of a job to compete.
-#
-
-replication.enabled=false
-
-# If replication is enabled, this value specifies the minimum time, in seconds,
-# a job needs to be queued in a batch queue in order to be considered for
-# replication
-#
-
-replication.min.queue.time=60
-
-# The maximum number of replicas that Swift should attempt.
-
-replication.limit=3
-
-#
-# WARNING: This option is deprecated. Please use the hostname option.
-#
-# The IP address of the submit machine is used by GRAM as a callback
-# address to report the status of submitted jobs. In general, Swift
-# can automatically detect the IP address of the local machine.
-# However, if the machine has more than one network interface, Swift
-# will pick the first one, which may not be the right choice. It is
-# recommended that this property is set properly before attempting to
-# run jobs through GRAM.
-#
-# Format:
-#    ip.address=x.y.z.w
-#
-
-#ip.address=127.0.0.1
-
-
-# Controls how Swift will communicate the result code of running user programs
-# from workers to the submit side. In files mode, a file
-# indicating success or failure will be created on the site shared filesystem.
-# In provider mode, the execution provider job status will
-# be used. Notably, GRAM2 does not return job statuses correctly, and so
-# provider mode will not work with GRAM2. With other
-# providers, it can be used to reduce the amount of filesystem access compared
-# to files mode.
-#
-# status.mode=files
-
-# Controls how swift will supply parameters to the remote wrapper script.
-# 'args' mode will pass parameters on the command line
-# 'files' mode will pass parameters through an additional input file
-#
-# valid values: args, files
-# Default: files
-#
-# wrapper.parameter.mode=args
-
-# Determines if Swift remote wrappers will be executed by specifying an
-# absolute path, or a path relative to the job initial working directory
-#
-# valid values: absolute, relative
-# wrapper.invocation.mode=absolute
-
-#
-# Limits the number of concurrent iterations that each foreach statement
-# can have at one time. This conserves memory for swift programs that
-# have large numbers of iterations (which would otherwise all be executed
-# in parallel).
-#
-# Default: 1024
-#
-
-foreach.max.threads=16384
-
-# controls whether the log file will contain provenance information
-# enabling this will increase the size of log files, sometimes
-# significantly.
-
-provenance.log=false
-
-# Controls whether file staging is done by swift or by the execution
-# provider. If set to false, the standard swift staging mechanism is
-# used. If set to true, swift does not stage files. Instead, the
-# execution provider is instructed to stage files in and out.
-#
-# Provider staging is experimental.
-#
-# When enabled, and when coasters are used as an execution provider,
-# a staging mechanism can be selected for each site
-# using the swift:stagingMethod site profile in sites.xml. The
-# following is a list of accepted mechanisms:
-#
-# * file:  Staging is done from a filesystem accessible to the
-#          coaster service (typically running on the head node)
-# * proxy: Staging is done from a filesystem accessible to the
-#          client machine that swift is running on, and is proxied
-#          through the coaster service
-# * sfs:   (short for "shared filesystem") Staging is done by
-#          copying files to and from a filesystem accessible
-#          by the compute node (such as an NFS or GPFS mount).
-
-
-use.provider.staging=false
-provider.staging.pin.swiftfiles=false



From hategan at ci.uchicago.edu  Mon Jun 11 13:38:26 2012
From: hategan at ci.uchicago.edu (hategan at ci.uchicago.edu)
Date: Mon, 11 Jun 2012 13:38:26 -0500 (CDT)
Subject: [Swift-commit] r5811 - trunk/src/org/griphyn/vdl/mapping
Message-ID: <20120611183826.CE5889CCE5@svn.ci.uchicago.edu>

Author: hategan
Date: 2012-06-11 13:38:26 -0500 (Mon, 11 Jun 2012)
New Revision: 5811

Modified:
   trunk/src/org/griphyn/vdl/mapping/ArrayDataNode.java
Log:
fix concurrent modification exception in array wrapper initialization

Modified: trunk/src/org/griphyn/vdl/mapping/ArrayDataNode.java
===================================================================
--- trunk/src/org/griphyn/vdl/mapping/ArrayDataNode.java	2012-06-11 17:43:58 UTC (rev 5810)
+++ trunk/src/org/griphyn/vdl/mapping/ArrayDataNode.java	2012-06-11 18:38:26 UTC (rev 5811)
@@ -107,12 +107,14 @@
     }
 
     @Override
-    protected synchronized Future getFutureWrapper() {
-    	if (wrapper == null) {
-    		wrapper = new ArrayIndexFutureList(this, this.getArrayValue());
-    		FutureTracker.get().add(this, wrapper);
-    	}
-        return wrapper;
+    protected Future getFutureWrapper() {
+        synchronized(getHandles()) {
+        	if (wrapper == null) {
+        		wrapper = new ArrayIndexFutureList(this, this.getArrayValue());
+        		FutureTracker.get().add(this, wrapper);
+        	}
+            return wrapper;
+        }
     }
 
     public FutureList getFutureList() {



From swift at ci.uchicago.edu  Wed Jun 13 00:05:04 2012
From: swift at ci.uchicago.edu (swift at ci.uchicago.edu)
Date: Wed, 13 Jun 2012 00:05:04 -0500 (CDT)
Subject: [Swift-commit] cog r3422
Message-ID: <20120613050504.D54748D00D86@bridled.ci.uchicago.edu>

------------------------------------------------------------------------
r3422 | davidkelly999 | 2012-06-13 00:02:12 -0500 (Wed, 13 Jun 2012) | 2 lines

More carefully check if a port range is specified

------------------------------------------------------------------------
Index: modules/provider-coaster/src/org/globus/cog/abstraction/coaster/service/LocalTCPService.java
===================================================================
--- modules/provider-coaster/src/org/globus/cog/abstraction/coaster/service/LocalTCPService.java	(revision 3421)
+++ modules/provider-coaster/src/org/globus/cog/abstraction/coaster/service/LocalTCPService.java	(working copy)
@@ -103,7 +103,9 @@
             channel.configureBlocking(true);
             if(port == 0) {
               PortRange portRange = PortRange.getTcpInstance();
-              port = portRange.getFreePort(port);
+              if(portRange != null && portRange.isEnabled()) {
+                  port = portRange.getFreePort(port);
+              }
             }
             channel.socket().bind(new InetSocketAddress(port));
             


From swift at ci.uchicago.edu  Wed Jun 13 11:55:03 2012
From: swift at ci.uchicago.edu (swift at ci.uchicago.edu)
Date: Wed, 13 Jun 2012 11:55:03 -0500 (CDT)
Subject: [Swift-commit] cog r3423
Message-ID: <20120613165504.9F8EF8D00D86@bridled.ci.uchicago.edu>

------------------------------------------------------------------------
r3423 | hategan | 2012-06-13 11:54:00 -0500 (Wed, 13 Jun 2012) | 1 line

fixed deadlock (some unnecessary sync) and re-added logging statements to log site scores (used by the TUI)
------------------------------------------------------------------------
Index: modules/karajan/src/org/globus/cog/karajan/scheduler/WeightedHostSet.java
===================================================================
--- modules/karajan/src/org/globus/cog/karajan/scheduler/WeightedHostSet.java	(revision 3422)
+++ modules/karajan/src/org/globus/cog/karajan/scheduler/WeightedHostSet.java	(working copy)
@@ -61,6 +61,24 @@
 			checkOverloaded(wh);
 		}
 	}
+	
+	public double factorScore(WeightedHost wh, double factor) {
+		synchronized (scores) {
+			scores.remove(wh);
+			sum -= wh.getTScore();
+			double newScore = factor(wh.getScore(), factor);
+			wh.setScore(newScore);
+			weightedHosts.put(wh.getHost(), wh);
+			scores.add(wh);
+			sum += wh.getTScore();
+			checkOverloaded(wh);
+			return newScore;
+		}
+	}
+	
+	protected final double factor(double score, double factor) {
+		return score + factor;
+	}
 
 	public void changeLoad(WeightedHost wh, int dl) {
 		wh.changeLoad(dl);
Index: modules/karajan/src/org/globus/cog/karajan/scheduler/WeightedHostScoreScheduler.java
===================================================================
--- modules/karajan/src/org/globus/cog/karajan/scheduler/WeightedHostScoreScheduler.java	(revision 3422)
+++ modules/karajan/src/org/globus/cog/karajan/scheduler/WeightedHostScoreScheduler.java	(working copy)
@@ -140,13 +140,12 @@
 	    return sorted;
 	}
 
-	protected synchronized void factorScore(WeightedHost wh, double factor) {
-		double score = wh.getScore();
+	protected void factorScore(WeightedHost wh, double factor) {
 		if (logger.isDebugEnabled()) {
 			logger.debug("multiplyScore(" + wh + ", " + factor + ")");
 		}
-		double ns = factor(score, factor);
-		sorted.changeScore(wh, ns);
+		double score = wh.getScore();
+		double ns = sorted.factorScore(wh, factor);
 		if (logger.isDebugEnabled()) {
 			logger.debug("Old score: " + WeightedHost.D4.format(score) + ", new score: "
 					+ WeightedHost.D4.format(ns));
@@ -160,10 +159,6 @@
 		wh.setDelayedDelta(wh.getDelayedDelta() + factor);
 	}
 
-	protected final double factor(double score, double factor) {
-		return score + factor;
-	}
-
 	protected synchronized BoundContact getNextContact(TaskConstraints t)
 			throws NoFreeResourceException {
 		checkGlobalLoadConditions();
@@ -204,14 +199,17 @@
 			throw new NoFreeResourceException();
 		}
 		else if (s.size() == 1) {
-			selected = (WeightedHost) s.iterator().next();
+			selected = s.iterator().next();
+			if (logger.isInfoEnabled()) {
+				logger.info("Sorted: " + s);
+			}
 		}
 		else {
 			double sum = s.getSum();
 			if (policy == POLICY_WEIGHTED_RANDOM) {
 				double rand = Math.random() * sum;
-				if (logger.isDebugEnabled() && !s.isEmpty()) {
-					logger.debug("Sorted: " + s);
+				if (logger.isInfoEnabled() && !s.isEmpty()) {
+					logger.info("Sorted: " + s);
 				}
 				if (logger.isDebugEnabled()) {
 					logger.debug("Rand: " + rand + ", sum: " + sum);


From hategan at ci.uchicago.edu  Sat Jun 23 15:40:32 2012
From: hategan at ci.uchicago.edu (hategan at ci.uchicago.edu)
Date: Sat, 23 Jun 2012 15:40:32 -0500 (CDT)
Subject: [Swift-commit] r5812 - trunk/src/org/griphyn/vdl/karajan/lib
Message-ID: <20120623204032.78C589CCBE@svn.ci.uchicago.edu>

Author: hategan
Date: 2012-06-23 15:40:31 -0500 (Sat, 23 Jun 2012)
New Revision: 5812

Modified:
   trunk/src/org/griphyn/vdl/karajan/lib/Execute.java
Log:
pass stage-in and stage-out status from provider

Modified: trunk/src/org/griphyn/vdl/karajan/lib/Execute.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/Execute.java	2012-06-11 18:38:26 UTC (rev 5811)
+++ trunk/src/org/griphyn/vdl/karajan/lib/Execute.java	2012-06-23 20:40:31 UTC (rev 5812)
@@ -110,6 +110,12 @@
 					RuntimeStats.setProgress(stack, "Submitted");
 					getReplicationManager(stack).submitted(task, e.getStatus().getTime());
 				}
+				else if (c == Status.STAGE_IN) {
+				    RuntimeStats.setProgress(stack, "Stage in");
+				}
+				else if (c == Status.STAGE_OUT) {
+				    RuntimeStats.setProgress(stack, "Stage out");
+				}
 				else if (c == Status.ACTIVE) {
 					RuntimeStats.setProgress(stack, "Active");
 					getReplicationManager(stack).active(task, e.getStatus().getTime());



From hategan at ci.uchicago.edu  Sat Jun 23 15:41:54 2012
From: hategan at ci.uchicago.edu (hategan at ci.uchicago.edu)
Date: Sat, 23 Jun 2012 15:41:54 -0500 (CDT)
Subject: [Swift-commit] r5813 - trunk/src/org/griphyn/vdl/karajan/lib
Message-ID: <20120623204154.EC0039CCBE@svn.ci.uchicago.edu>

Author: hategan
Date: 2012-06-23 15:41:54 -0500 (Sat, 23 Jun 2012)
New Revision: 5813

Modified:
   trunk/src/org/griphyn/vdl/karajan/lib/InFileDirs.java
   trunk/src/org/griphyn/vdl/karajan/lib/OutFileDirs.java
   trunk/src/org/griphyn/vdl/karajan/lib/VDLFunction.java
Log:
when using generic url for files, use the host name as a path prefix to avoid clashes between files with the same name on different hosts

Modified: trunk/src/org/griphyn/vdl/karajan/lib/InFileDirs.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/InFileDirs.java	2012-06-23 20:40:31 UTC (rev 5812)
+++ trunk/src/org/griphyn/vdl/karajan/lib/InFileDirs.java	2012-06-23 20:41:54 UTC (rev 5813)
@@ -44,17 +44,24 @@
         VariableArguments ret = ArgUtil.getVariableReturn(stack);
         for (Object f : files) { 
         	String path = (String) f;
-            String dir = new AbsFile(path).getDir();
-            // there could be a clash here since
-            // "/a/b/c.txt" would be remotely the same
-            // as "a/b/c.txt". Perhaps absolute paths
-            // should have a unique prefix.
-            if (dir.startsWith("/") && dir.length() != 1) {
-            	ret.append(dir.substring(1));
-            }
-            else if (dir.length() != 0) {
-                ret.append(dir);
-            }
+        	AbsFile af = new AbsFile(path);
+        	if ("file".equals(af.getProtocol())) {
+                String dir = af.getDir();
+                // there could be a clash here since
+                // "/a/b/c.txt" would be remotely the same
+                // as "a/b/c.txt". Perhaps absolute paths
+                // should have a unique prefix.
+                if (dir.startsWith("/") && dir.length() != 1) {
+                	ret.append(dir.substring(1));
+                }
+                else if (dir.length() != 0) {
+                    ret.append(dir);
+                }
+        	}
+        	else {
+        	    // also prepend host name to the path
+        	    ret.append(af.getHost() + "/" + af.getDir());
+        	}
         }
         super.post(stack);
     }

Modified: trunk/src/org/griphyn/vdl/karajan/lib/OutFileDirs.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/OutFileDirs.java	2012-06-23 20:40:31 UTC (rev 5812)
+++ trunk/src/org/griphyn/vdl/karajan/lib/OutFileDirs.java	2012-06-23 20:41:54 UTC (rev 5813)
@@ -50,12 +50,18 @@
                 DSHandle handle = (DSHandle) pv.get(1);
                 DSHandle leaf = handle.getField(p);
                 String fname = VDLFunction.filename(leaf)[0];
-                String dir = new AbsFile(fname).getDir();
-                if (dir.startsWith("/") && dir.length() != 1) {
-                    ret.append(dir.substring(1));
+                AbsFile af = new AbsFile(fname);
+                if ("file".equals(af.getProtocol())) {
+                    String dir = af.getDir();
+                    if (dir.startsWith("/") && dir.length() != 1) {
+                        ret.append(dir.substring(1));
+                    }
+                    else if (dir.length() != 0) {
+                        ret.append(dir);
+                    }
                 }
-                else if (dir.length() != 0) {
-                    ret.append(dir);
+                else {
+                	ret.append(af.getHost() + "/" + af.getDir());
                 }
             }
         }

Modified: trunk/src/org/griphyn/vdl/karajan/lib/VDLFunction.java
===================================================================
--- trunk/src/org/griphyn/vdl/karajan/lib/VDLFunction.java	2012-06-23 20:40:31 UTC (rev 5812)
+++ trunk/src/org/griphyn/vdl/karajan/lib/VDLFunction.java	2012-06-23 20:41:54 UTC (rev 5813)
@@ -280,7 +280,13 @@
 	}
 
 	protected static String pathOnly(String file) {
-		return new AbsFile(file).getPath();
+	    AbsFile af = new AbsFile(file);
+	    if ("file".equals(af.getProtocol())) {
+	        return af.getPath();
+	    }
+	    else {
+	        return af.getHost() + "/" + af.getPath();
+	    }
 	}
 
 	protected String[] pathOnly(String[] files) {



From hategan at ci.uchicago.edu  Sat Jun 23 15:44:26 2012
From: hategan at ci.uchicago.edu (hategan at ci.uchicago.edu)
Date: Sat, 23 Jun 2012 15:44:26 -0500 (CDT)
Subject: [Swift-commit] r5814 - trunk/libexec
Message-ID: <20120623204426.23D389CCBE@svn.ci.uchicago.edu>

Author: hategan
Date: 2012-06-23 15:44:26 -0500 (Sat, 23 Jun 2012)
New Revision: 5814

Modified:
   trunk/libexec/_swiftwrap.wrapperstaging
   trunk/libexec/vdl-int-wrapper-staging.k
Log:
uniform file naming convention for wrapper staging (fixes status.mode=files for wrapper staging); properly deal with URLs that are not automatically prefixed using URLPREFIX

Modified: trunk/libexec/_swiftwrap.wrapperstaging
===================================================================
--- trunk/libexec/_swiftwrap.wrapperstaging	2012-06-23 20:41:54 UTC (rev 5813)
+++ trunk/libexec/_swiftwrap.wrapperstaging	2012-06-23 20:44:26 UTC (rev 5814)
@@ -55,23 +55,47 @@
 	[[ $DEBUG == 1 ]] && echo "$@" >& "$INFO"
 }
 
+localPath() {
+	# remove protocol://
+	PATH="${1#*://}"
+	# remove leading "/" if present
+	PATH="${PATH#/}"
+	log "Transformed $1 to $PATH"
+	echo $PATH
+}
+
 stagein() {
 	SRC=$1
 	DST=$2
 	
-	log "Staging in $URLPREFIX/$SRC to $DST"
+	case $SRC in
+		*://*)
+			;;
+		*)
+			SRC=$URLPREFIX/$SRC
+			;;
+	esac
 	
-	case $URLPREFIX in
+	log "Staging in $SRC to $DST"
+	
+	case $SRC in
 		file://*)
-			SRC2="${URLPREFIX#file://}/$SRC"
-			log "Copying $SRC2 to $DST"
-			cp $SRC2 $DST
+			SRC=${SRC#file://}
+			log "Copying $SRC to $DST"
+			if [ ! -f $SRC ]; then
+				fail 254 "Cannot stage in $SRC. File not found."
+			fi
+			cp $SRC $DST 2>&1 >& "$INFO"
 			;;
+		http://*)
+			log "Fetching $SRC using wget"
+			wget $SRC -O $DST 2>&1 >& "$INFO"
+			;;
 		*://*)
-			fail 254 "Cannot handle protocol ${URLPREFIX%%://*}"
+			fail 254 "Cannot handle protocol ${SRC%%://*}"
 			;;
 		*)
-			fail 254 "Invalid URL prefix: $URLPREFIX"
+			fail 254 "Invalid URL: $SRC"
 			;;
 	esac
 }
@@ -80,21 +104,30 @@
 	SRC=$1
 	DST=$2
 	
-	log "Staging out $SRC to $URLPREFIX/$DST"
+	case $DST in
+		*://*)
+			;;
+		*)
+			DST=$URLPREFIX/$DST
+	esac
 	
-	case $URLPREFIX in
+	log "Staging out $SRC to $DST"
+	
+	local DIR
+	
+	case $DST in
 		file://*)
-			DST2="${URLPREFIX#file://}/$DST"
-			DIR=`dirname $DST2`
+			DST=${DST#file://}
+			DIR=`dirname $DST`
 			mkdir -p $DIR
-			log "Copying $SRC to $DST2"
-			cp $SRC $DST2
+			log "Copying $SRC to $DST"
+			cp $SRC $DST 2>&1 >& "$INFO"
 			;;
 		*://*)
-			fail 254 "Cannot handle protocol ${URLPREFIX%%://*}"
+			fail 254 "Cannot handle protocol ${DST%%://*}"
 			;;
 		*)
-			fail 254 "Invalid URL prefix: $URLPREFIX"
+			fail 254 "Invalid URL: $DST"
 			;;
 	esac
 }
@@ -105,8 +138,8 @@
 	shift
 	
 	if [ "X$DIR" != "X" ]; then
-		echo $@ >"$DIR/_error"
-		stageout "$DIR/_error" "$JOBDIR/error"
+		echo $@ >"$DIR/_swift.error"
+		stageout "$DIR/_swift.error" "$JOBDIR/_swift.error"
 	else
 		echo $@
 	fi
@@ -266,12 +299,12 @@
 closeinfo
 
 if [ -z $MPI_RANK ]; then
-	INFOFILE="$INFODIR/_info"
+	INFOFILE="$INFODIR/_swift.info"
 else
 	# Rename info file for each rank
-	INFOFILE="$INFODIR/_info-${PMI_RANK}"
+	INFOFILE="$INFODIR/_swift.info-${PMI_RANK}"
 	# Build list of per-rank info files
-	echo $INFOFILE >> $INFODIR/_info
+	echo $INFOFILE >> $INFODIR/_swift.info
 fi
 rm -f $INFOFILE
 openinfo "$INFOFILE"
@@ -421,7 +454,7 @@
 			continue
 		fi
 		
-		stagein $L "$DIR/$L"
+		stagein $L "$DIR/`localPath $L`"
 	done
 	
 	if [[ $CDM_FILE != "" ]]; then
@@ -484,11 +517,12 @@
 
 	MISSING=
 	for O in $OUTF ; do
-		if [ ! -f "$DIR/$O" ]; then
+		LO=`localPath $O`
+		if [ ! -f "$DIR/$LO" ]; then
 			if [ "$MISSING" == "" ]; then
-				MISSING=$O
+				MISSING=$LO
 			else
-				MISSING="$MISSING, $O"
+				MISSING="$MISSING, $LO"
 			fi
 		fi
 	done
@@ -500,7 +534,7 @@
 	logstate "MOVING_OUTPUTS $OUTF"
 	for O in $OUTF ; do
 		if ! contains SKIPPED_OUTPUT $O ; then
-			stageout "$DIR/$O" "$O" 
+			stageout "$DIR/`localPath $O`" "$O" 
 		fi
 	done
 	
@@ -508,9 +542,9 @@
 	cdm_gather $GATHER_OUTPUT
 	
 	if [ "$STATUSMODE" = "files" ]; then
-		logstate "TOUCH_SUCCESS"
-		touch ${ID}-success
-		stageout "$DIR/${ID}-success" "$JOBDIR/success"  
+		logstate "TOUCH_SUCCESS $DIR `pwd`"
+		touch _swift.success
+		stageout "$DIR/_swift.success" "$JOBDIR/_swift.success"  
 	fi
 	
 	log "Moving back to workflow directory $WFDIR"
@@ -520,7 +554,7 @@
 	fi
 	
 	if [ "$WRAPPERLOG_ALWAYS_TRANSFER" == "true" ]; then
-		stageout "$INFOFILE" "$JOBDIR/info"
+		stageout "$INFOFILE" "$JOBDIR/_swift.info"
 	fi
 	if [ "$SITEDIR_KEEP" != "true" ]; then
 		logstate "RM_JOBDIR"
@@ -537,7 +571,7 @@
 closeinfo
 
 if [ "$WRAPPER_LOG_ALWAYS_TRANSFER" == "true" ]; then
-	stageout "$DIR/${ID}-info" "$JOBID/${ID}-info"
+	stageout "$DIR/_swift.info" "$JOBID/_swift.info"
 fi
 
 # ensure we exit with a 0 after a successful execution

Modified: trunk/libexec/vdl-int-wrapper-staging.k
===================================================================
--- trunk/libexec/vdl-int-wrapper-staging.k	2012-06-23 20:41:54 UTC (rev 5813)
+++ trunk/libexec/vdl-int-wrapper-staging.k	2012-06-23 20:44:26 UTC (rev 5814)
@@ -22,25 +22,7 @@
 
 namespace("vdl"
 	export(
-		element(rmdir, [dir, host]
-			parallelFor(entry, file:list(dir, host=host)
-				epath := "{dir}/{entry}"
-				if(
-					file:isDirectory(epath, host=host) rmdir(epath, host)
-					file:remove(epath, host=host)
-				)
-			)
-			dir:remove(dir, host=host)
-		)
-
-		element(createdirs, [path, dir, host]
-			dc := dircat(dir, path)
-			log(LOG:INFO, "START path={path} dir={dir} - Creating directory structure")
-
-			dir:make(dc, host=host)
-		)
-
-		element(checkJobStatus, [wfdir, jobid, tr]
+		element(checkJobStatus, [jobdir, jobid, tr]
 			log(LOG:DEBUG, "START jobid={jobid}")
 			try(
 				sequential(
@@ -51,39 +33,24 @@
 					 * in one operation. It relies on file:remove() throwing
 					 * an exception if the file is not there.
 					 */
-					file:remove("{wfdir}/status/{jobdir}/{jobid}-success", host=rhost)
+					file:remove("{jobdir}/_swift.success")
 					log(LOG:INFO, "SUCCESS jobid={jobid} - Success file found")
 				)
-				sequential(
-					try (
-						msg = checkErrorFile(rhost, wfdir, jobid, jobdir)
-						sequential (
-							log(LOG:INFO, "NO_STATUS_FILE jobid={jobid} - Both status files are missing")
-							throw("No status file was found. Check the shared filesystem on {rhost}")
-						)
-					)
-				)
-				throw(checkErrorFile(rhost, wfdir, jobid, jobdir))
+				throw(checkErrorFile(jobdir, jobid))
 			)
 		)
 		
-		element(checkErrorFile, [wfdir, jobid]
+		element(checkErrorFile, [jobdir, jobid]
 			if (
-				file:exists("{wfdir}/status/{jobdir}/{jobid}-error", host=rhost) then(
+				file:exists("{jobdir}/_swift.error") then(
 					log(LOG:INFO, "FAILURE jobid={jobid} - Failure file found")
-					task:transfer(srchost=rhost, srcdir="{wfdir}/status/{jobdir}", srcfile="{jobid}-error")
-					error := parallel(
-						file:remove("{wfdir}/status/{jobdir}/{jobid}-error", host=rhost)
-						sequential(
-							str:strip(file:read("{jobid}-error"))
-							file:remove("{jobid}-error")
-						)
-					)
+					error := str:strip(file:read("{jobdir}/_swift.error"))
+					file:remove("{jobdir}/_swift.error")
 					error
 				)
 				else (
-					log(LOG:INFO, "NO_STATUS_FILE jobid={jobid} - Error file missing")
-					throw("No status file was found. Check the shared filesystem on {rhost}")
+					log(LOG:INFO, "NO_STATUS_FILE jobid={jobid} - Both status files are missing")
+					throw("No status file was found")
 				)
 			)
 		)
@@ -116,7 +83,7 @@
 		)
 
 		element(inFiles, [stageins]
-			pathnames(stageins)
+			stageins
 		)
 
 		element(fileDirs, [stageins, stageouts]
@@ -128,42 +95,8 @@
 			)
 		)
 
-		element(createDirSet, [jobid, destdir, host, dirs]
-			/*
-			 * Ideally this would be done by creating a tree of the directories
-			 * to be created and (eventually) exploiting the concurrency in that.
-			 */
-			log(LOG:INFO, "START jobid={jobid} host={host} - Initializing directory structure")
-			for(u, dirs
-				cacheOn(list(u, destdir, host)
-					createdirs(u, destdir, host)
-				)
-			)
-			log(LOG:INFO, "END jobid={jobid} - Done initializing directory structure")
-		)
-
 		element(cleanup, [dir, host]
 			log(LOG:INFO, "START dir={dir} host={host}")
-			cdmfile := cdm:file()
-			log(LOG:DEBUG, "cdmfile {cdmfile}")
-			if(cdmfile != "" &
-                           cdm:get("GATHER_DIR") != "UNSET" then(
-				log(LOG:INFO, "submitting cdm_cleanup.sh to {dir}")
-				task:transfer(srcfile="cdm_cleanup.sh",
-					      srcdir="{swift.home}/libexec",
-					      desthost=host, destdir=dir)
-				task:transfer(srcfile="cdm_lib.sh",
-					      srcdir="{swift.home}/libexec",
-					      desthost=host, destdir=dir)
-			        log(LOG:INFO, "execute: cdm_cleanup.sh")
-				task:execute(
-					     executable="/bin/bash",
-					     arguments=list("{dir}/cdm_cleanup.sh",
-					                     cdm:get("GATHER_DIR"), cdm:get("GATHER_TARGET")
-							     sys:uid() )
-					     host=host, batch=true, tcprofile(host))
-				)
-			)
 			if(vdl:configProperty("sitedir.keep") == "false"
 				task:execute(
 					vdl:siteprofile(host, "swift:cleanupCommand"),
@@ -191,14 +124,6 @@
 			log(LOG:INFO, "END cleanups={cleanup}")
 		)
 
-		element(cleanupFiles, [files, host]
-			uParallelFor(r, files
-				log(LOG:INFO, "Purging ", r, " on ", host)
-				file:remove(r, host=host)
-				vdl:cacheFileRemoved(r, host)
-			)
-		)
-
 		element(stageWrapperParams, [jobid, wrapfile, dir, host]
 			log(LOG:INFO, "START jobid={jobid} - staging in wrapper params")
 			provider := provider(wrapfile)
@@ -243,16 +168,10 @@
 				)
 			)
 		)
-
-		element(fileSizes, [files]
-			math:sum(
-				for(f, files, file:size(file))
-			)
-		)
 		
 		element(readStandardFiles, [jobdir, stdout, stderr]
 			concat(
-				for(f, list(list("stderr.txt", stderr), list("stdout.txt", stdout))
+				for(f, list(list("_swift.stderr", stderr), list("_swift.stdout", stdout))
 					[name, file] := each(f)
 					destfile := "{jobdir}/{file}"
 					nl()
@@ -291,8 +210,8 @@
 
 				wrapfile := "{jobdir}/_paramfile"
 
-				stdout := try(getFieldValue(stdout), "stdout.txt")
-				stderr := try(getFieldValue(stderr), "stderr.txt")
+				stdout := try(getFieldValue(stdout), "_swift.stdout")
+				stderr := try(getFieldValue(stderr), "_swift.stderr")
 				fileDirs := fileDirs(stagein, stageout)
 				os := vdl:siteprofile(rhost, "SYSINFO:OS")
 				
@@ -306,8 +225,8 @@
 							"-err ", stderr, nl(),
 							"-i ", maybe(getFieldValue(stdin)), nl(),
 							"-d ", flatten(each(fileDirs)), nl(),
-							"-if ", flatten(infiles(stagein)), nl(),
-							"-of ", flatten(outfiles(stageout)), nl(),
+							"-if ", flatten(inFiles(stagein)), nl(),
+							"-of ", flatten(outFiles(stageout)), nl(),
 							"-wt", WRAPPERLOG_ALWAYS_TRANSFER,
 							"-sk", SITEDIR_KEEP,
 							"-cdmfile ", cdm:file(), nl(),
@@ -366,8 +285,8 @@
 									"-err", stderr,
 									"-i", maybe(getFieldValue(stdin)),
 									"-d", flatten(each(fileDirs)),
-									"-if", flatten(infiles(stagein)),
-									"-of", flatten(outfiles(stageout)),
+									"-if", flatten(inFiles(stagein)),
+									"-of", flatten(outFiles(stageout)),
 									"-wt", WRAPPERLOG_ALWAYS_TRANSFER,
 									"-sk", SITEDIR_KEEP,
 									"-cdmfile", cdm:file(),
@@ -386,7 +305,7 @@
 
 						vdl:setprogress("Checking status")
 						if(statusMode == "files"
-							checkJobStatus(wfdir, jobid, tr)
+							checkJobStatus(jobdir, jobid, tr)
 						)
 
 						if(wrapperMode == "files"
@@ -412,7 +331,7 @@
 					catch("^(?!Abort$).*"
 						vdl:setprogress("Failed but can retry")
 						prev := exception
-						exception := try(exception(checkErrorFile(rhost, wfdir, jobid)), prev)
+						exception := try(exception(checkErrorFile(jobdir, jobid)), prev)
 						
 						log(LOG:DEBUG, "APPLICATION_EXCEPTION jobid={jobid} - Application exception: ", exception)