[Swift-commit] r5555 - SwiftApps/SciColSim

[ketan at ci.uchicago.edu]

Author: ketan
Date: 2012-02-07 15:47:30 -0600 (Tue, 07 Feb 2012)
New Revision: 5555

Added:
   SwiftApps/SciColSim/RunSwift.sh
   SwiftApps/SciColSim/start-mcs-workers
Log:
adding mcs workers running script

Added: SwiftApps/SciColSim/RunSwift.sh
===================================================================
--- SwiftApps/SciColSim/RunSwift.sh	                        (rev 0)
+++ SwiftApps/SciColSim/RunSwift.sh	2012-02-07 21:47:30 UTC (rev 5555)
@@ -0,0 +1,49 @@
+#!/bin/bash
+
+# ./Runswift local to run on sandbox
+# ./Runswift clustersmall to run on beagle pbs cluster at small scale
+# ./Runswift clusterquick to run on beagle pbs cluster at large scale
+
+escapecode=$(echo -n -e '\033')
+
+count=$(head -1 counter.txt);
+expr $count + 1 > counter.txt
+mkdir run$count
+cp /home/ketan/SciColSim/*.swift run$count/
+cp /home/ketan/SciColSim/sites.beagle.xml run$count/
+cp /home/ketan/SciColSim/sites.beagle.quick.xml run$count/
+cp local.xml run$count/
+cp /home/ketan/SciColSim/tc run$count/
+cp /home/ketan/SciColSim/movie_graph.txt run$count/
+cp /home/ketan/SciColSim/cf run$count/
+cd run$count
+
+if [ $1 = "local" ]
+then
+ #SWIFT_HEAP_MAX=7000M swift -tc.file tc -sites.file local.xml -config cf annealing.swift -e33="$escapecode" -nworkers=36 >& swift.out
+ #Total jobs = 6 * 1 * 120/20 * 3 * 100 = 10,800
+ SWIFT_HEAP_MAX=7000M swift -tc.file tc -sites.file local.xml -config cf annealing.swift -e33="$escapecode" -nworkers=6 -minrange=58 -maxrange=64 -rangeinc=1 -evoreruns=120 -nreps=1 -alphai=0 -alpham=0 -beta=4.0 -gamma=50.0 -delta=-1 -annealingcycles=100 -rerunsperapp=20  >& swift.out
+
+elif [ $1 = "clusterbig" ]
+then
+ SWIFT_HEAP_MAX=7000M swift -tc.file tc -sites.file sites.beagle.xml -config cf annealing.swift -e33="$escapecode" -nworkers=24 -rangeinc=50 -evoreruns=960 -startingjump=2.3 -alphai=0 -alpham=0 -beta=4.0 -gamma=50.0 -delta=-1 -annealingcycles=100 -rerunsperapp=192  >& swift.out
+
+elif [ $1 = "clustersmall" ]
+then
+ SWIFT_HEAP_MAX=7000M swift -tc.file tc -sites.file sites.beagle.xml -config cf annealing.swift \-e33="$escapecode" \
+      >& swift.out
+
+elif [ $1 = "clusterquick" ]
+then
+#target_innovation=(1009-58)/50=~20
+#repeats=nreps=1
+# 3 repeats constant (serial)
+#annealing_cycles=100 (serial)
+#rerunsperapp=192
+#evoreruns=960
+#J=evoreruns/rerunsperapp=960/192=5
+
+#Total parallel jobs = (maxrange-minrange)/rangeinc * nreps * (evoreruns/rerunsperapp) = (1009-58)/50 * 1 * 960/192 = 20*5 = 100 Jobs = 2400 openmp jobs in parallel
+ SWIFT_HEAP_MAX=7000M swift -tc.file tc -sites.file sites.beagle.quick.xml -config cf annealing.swift -e33="$escapecode" -nworkers=24 -minrange=58 -maxrange=1009 -rangeinc=50 -evoreruns=960 -nreps=1 -alphai=0 -alpham=0 -beta=4.0 -gamma=50.0 -delta=-1 -annealingcycles=100 -rerunsperapp=192  >& swift.out
+fi
+


Property changes on: SwiftApps/SciColSim/RunSwift.sh
___________________________________________________________________
Added: svn:executable
   + *

Added: SwiftApps/SciColSim/start-mcs-workers
===================================================================
--- SwiftApps/SciColSim/start-mcs-workers	                        (rev 0)
+++ SwiftApps/SciColSim/start-mcs-workers	2012-02-07 21:47:30 UTC (rev 5555)
@@ -0,0 +1,42 @@
+#! /bin/bash
+
+#if [ $# -gt 0 ]; then
+#  COMPUTEHOSTS=$*
+#fi
+
+#for h in $COMPUTEHOSTS; do
+#  echo Host: $h
+#  ssh $h rm -rf /tmp/$USER
+#  if [ $(expr $h : -) != 0 ]; then
+#    echo $0: Error: invalid argument or host name: $h
+#    exit 1
+#  fi
+#done
+
+COMPUTEHOSTS='crush thwomp stomp crank steamroller grind churn trounce thrash vanquish'
+
+SWIFTBIN=/home/ketan/swift-install/0.93/cog/modules/swift/dist/swift-svn/bin
+#SERVICE_URL="http://140.221.8.75:35753"
+SERVICE_URL="$1"
+
+LOGDIR=/tmp/$USER/Swift/workers
+
+IDLETIMEOUT=$((60*60*240)) # 10 days: FIXME: make this a command line arg
+
+
+for host in $(echo $COMPUTEHOSTS); do
+   timestamp=$(date "+%Y.%m%d.%H%M%S")
+   random=$(awk "BEGIN {printf \"%0.5d\", $RANDOM}")
+   ID=$timestamp.$random
+    # FIXME: make logging an argument; set false by default
+    # fixme:send worker.pl to remote host via stdin or scp.
+    ssh $host /bin/sh -c \'"mkdir -p $LOGDIR"\'
+    scp $SWIFTBIN/worker.pl $host:$LOGDIR
+      
+   ssh $host '/bin/sh -c '\'"WORKER_LOGGING_LEVEL=DEBUG $LOGDIR/worker.pl $SERVICE_URL $ID $LOGDIR 2>&1 & echo PID=\$!"\'  >remotepid.$host </dev/null &
+          sshpids="$sshpids $!"
+      #done
+  done
+
+echo Started workers from these ssh processes: $sshpids
+#echo $sshpids > $sshpidfile


Property changes on: SwiftApps/SciColSim/start-mcs-workers
___________________________________________________________________
Added: svn:executable
   + *