[Swift-commit] r6481 - demo/cray/swift-1.0/part08-cray-loop
wilde at ci.uchicago.edu
wilde at ci.uchicago.edu
Thu Apr 18 13:24:07 CDT 2013
Author: wilde
Date: 2013-04-18 13:24:07 -0500 (Thu, 18 Apr 2013)
New Revision: 6481
Added:
demo/cray/swift-1.0/part08-cray-loop/beagle.xml
demo/cray/swift-1.0/part08-cray-loop/raven-direct.xml
demo/cray/swift-1.0/part08-cray-loop/raven.xml
Removed:
demo/cray/swift-1.0/part08-cray-loop/cray.xml
demo/cray/swift-1.0/part08-cray-loop/pbs.xml
demo/cray/swift-1.0/part08-cray-loop/raven.xml
demo/cray/swift-1.0/part08-cray-loop/ravenps.xml
Modified:
demo/cray/swift-1.0/part08-cray-loop/README
Log:
Adjust README and sites file names.
Modified: demo/cray/swift-1.0/part08-cray-loop/README
===================================================================
--- demo/cray/swift-1.0/part08-cray-loop/README 2013-04-18 18:12:35 UTC (rev 6480)
+++ demo/cray/swift-1.0/part08-cray-loop/README 2013-04-18 18:24:07 UTC (rev 6481)
@@ -36,21 +36,29 @@
$ swift p8.swift -nsim=10 -steps=5 -range=1000 -count=5
+
!!! STOP HERE - remaining needs adjustments
+
+PART 2:
+
Next, you can experiment with different size runs:
-Adjust the job sizes:
+Adjust the job sizes in sites.xml:
<profile namespace="globus" key="slots">6</profile>
<profile namespace="globus" key="nodeGranularity">4</profile>
<profile namespace="globus" key="maxNodes">4</profile>
+TODO: Explain the above!!!
+PART 3:
+Next, you can experiment with different data management strategy
+
In these runs, Swift was moving data to and from the data/ directory by
staging to the compute node local (RAM) filesystem. For some workflows its
better to bypass staging and access all input and output files directly.
@@ -59,17 +67,13 @@
and turning off staging to the local filesystem:
$ sed -i -e '/SET PROVIDER STAGING/s/=true/=false/p' $HOME/.swift/swift.properties
+ $ cp raven-direct.xml sites.xml
Now run the script with "direct" data acces
- swift -sites.file ravenps.xml -cdm.file dmrules p8.swift -steps=1 -range=1000 -count=6 -nsim=10
+ swift -cdm.file dmrules p8.swift -steps=1 -range=1000 -count=6 -nsim=10
-
-
-The app list
-
-
OLD: FIXME:
TOPIC: Run the "simulations" on the parallel cluster
Copied: demo/cray/swift-1.0/part08-cray-loop/beagle.xml (from rev 6480, demo/cray/swift-1.0/part08-cray-loop/cray.xml)
===================================================================
--- demo/cray/swift-1.0/part08-cray-loop/beagle.xml (rev 0)
+++ demo/cray/swift-1.0/part08-cray-loop/beagle.xml 2013-04-18 18:24:07 UTC (rev 6481)
@@ -0,0 +1,31 @@
+<config>
+
+ <pool handle="localhost">
+ <execution provider="local"/>
+ <filesystem provider="local"/>
+ <workdirectory>/scratch/midway/{env.USER}/swiftwork</workdirectory>
+ </pool>
+
+ <pool handle="cray">
+ <execution provider="coaster" jobmanager="local:pbs"/>
+ <!-- <profile namespace="globus" key="project">CI-CCR000013</profile> -->
+
+ <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
+ <!-- <profile namespace="globus" key="queue">scalability</profile> -->
+ <profile namespace="globus" key="queue">batch</profile>
+
+ <profile namespace="globus" key="jobsPerNode">24</profile>
+ <profile namespace="globus" key="maxTime">1800</profile>
+
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="globus" key="maxNodes">1</profile>
+
+ <profile namespace="karajan" key="jobThrottle">12</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+
+ <filesystem provider="local"/>
+ <workdirectory>/lustre/beagle/wilde/swift/lab/swiftwork</workdirectory>
+ </pool>
+
+</config>
Deleted: demo/cray/swift-1.0/part08-cray-loop/cray.xml
===================================================================
--- demo/cray/swift-1.0/part08-cray-loop/cray.xml 2013-04-18 18:12:35 UTC (rev 6480)
+++ demo/cray/swift-1.0/part08-cray-loop/cray.xml 2013-04-18 18:24:07 UTC (rev 6481)
@@ -1,31 +0,0 @@
-<config>
-
- <pool handle="localhost">
- <execution provider="local"/>
- <filesystem provider="local"/>
- <workdirectory>/scratch/midway/{env.USER}/swiftwork</workdirectory>
- </pool>
-
- <pool handle="cray">
- <execution provider="coaster" jobmanager="local:pbs"/>
- <!-- <profile namespace="globus" key="project">CI-CCR000013</profile> -->
-
- <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
- <!-- <profile namespace="globus" key="queue">scalability</profile> -->
- <profile namespace="globus" key="queue">batch</profile>
-
- <profile namespace="globus" key="jobsPerNode">24</profile>
- <profile namespace="globus" key="maxTime">1800</profile>
-
- <profile namespace="globus" key="slots">1</profile>
- <profile namespace="globus" key="nodeGranularity">1</profile>
- <profile namespace="globus" key="maxNodes">1</profile>
-
- <profile namespace="karajan" key="jobThrottle">12</profile>
- <profile namespace="karajan" key="initialScore">10000</profile>
-
- <filesystem provider="local"/>
- <workdirectory>/lustre/beagle/wilde/swift/lab/swiftwork</workdirectory>
- </pool>
-
-</config>
Deleted: demo/cray/swift-1.0/part08-cray-loop/pbs.xml
===================================================================
--- demo/cray/swift-1.0/part08-cray-loop/pbs.xml 2013-04-18 18:12:35 UTC (rev 6480)
+++ demo/cray/swift-1.0/part08-cray-loop/pbs.xml 2013-04-18 18:24:07 UTC (rev 6481)
@@ -1,23 +0,0 @@
-<config>
- <pool handle="pbs">
- <execution provider="coaster" jobmanager="local:pbs"/>
- <!-- <profile namespace="globus" key="project">CI-CCR000013</profile> -->
-
- <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
- <!-- <profile namespace="globus" key="queue">scalability</profile> -->
- <profile namespace="globus" key="queue">batch</profile>
-
- <profile namespace="globus" key="jobsPerNode">24</profile>
- <profile namespace="globus" key="maxTime">1800</profile>
-
- <profile namespace="globus" key="slots">1</profile>
- <profile namespace="globus" key="nodeGranularity">1</profile>
- <profile namespace="globus" key="maxNodes">1</profile>
-
- <profile namespace="karajan" key="jobThrottle">12</profile>
- <profile namespace="karajan" key="initialScore">10000</profile>
-
- <filesystem provider="local"/>
- <workdirectory>/lustre/beagle/wilde/swift/lab/swiftwork</workdirectory>
- </pool>
-</config>
Copied: demo/cray/swift-1.0/part08-cray-loop/raven-direct.xml (from rev 6480, demo/cray/swift-1.0/part08-cray-loop/raven.xml)
===================================================================
--- demo/cray/swift-1.0/part08-cray-loop/raven-direct.xml (rev 0)
+++ demo/cray/swift-1.0/part08-cray-loop/raven-direct.xml 2013-04-18 18:24:07 UTC (rev 6481)
@@ -0,0 +1,35 @@
+<config>
+
+ <pool handle="localhost">
+ <execution provider="local"/>
+ <filesystem provider="local"/>
+ <workdirectory>{env.PWD}/swiftwork</workdirectory>
+ </pool>
+
+ <pool handle="cray">
+ <execution provider="coaster" jobmanager="local:pbs"/>
+ <!-- <profile namespace="globus" key="project">CI-CCR000013</profile> -->
+
+ <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=32</profile>
+ <!-- <profile namespace="globus" key="queue">scalability</profile> -->
+ <!-- <profile namespace="globus" key="queue">batch</profile> -->
+ <profile namespace="globus" key="queue">small</profile>
+
+ <profile namespace="globus" key="jobsPerNode">32</profile>
+ <profile namespace="globus" key="maxTime">1800</profile>
+
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="globus" key="maxNodes">1</profile>
+
+ <profile namespace="karajan" key="jobThrottle">6.00</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+
+ <profile namespace="env" key="PATHPREFIX">{env.PATHPREFIX}</profile>
+
+ <filesystem provider="local"/>
+ <workdirectory>{env.PWD}/swiftwork</workdirectory>
+
+ </pool>
+
+</config>
Deleted: demo/cray/swift-1.0/part08-cray-loop/raven.xml
===================================================================
--- demo/cray/swift-1.0/part08-cray-loop/raven.xml 2013-04-18 18:12:35 UTC (rev 6480)
+++ demo/cray/swift-1.0/part08-cray-loop/raven.xml 2013-04-18 18:24:07 UTC (rev 6481)
@@ -1,35 +0,0 @@
-<config>
-
- <pool handle="localhost">
- <execution provider="local"/>
- <filesystem provider="local"/>
- <workdirectory>{env.PWD}/swiftwork</workdirectory>
- </pool>
-
- <pool handle="cray">
- <execution provider="coaster" jobmanager="local:pbs"/>
- <!-- <profile namespace="globus" key="project">CI-CCR000013</profile> -->
-
- <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=32</profile>
- <!-- <profile namespace="globus" key="queue">scalability</profile> -->
- <!-- <profile namespace="globus" key="queue">batch</profile> -->
- <profile namespace="globus" key="queue">small</profile>
-
- <profile namespace="globus" key="jobsPerNode">32</profile>
- <profile namespace="globus" key="maxTime">1800</profile>
-
- <profile namespace="globus" key="slots">1</profile>
- <profile namespace="globus" key="nodeGranularity">1</profile>
- <profile namespace="globus" key="maxNodes">1</profile>
-
- <profile namespace="karajan" key="jobThrottle">6.00</profile>
- <profile namespace="karajan" key="initialScore">10000</profile>
-
- <profile namespace="env" key="PATHPREFIX">{env.PATHPREFIX}</profile>
-
- <filesystem provider="local"/>
- <workdirectory>{env.PWD}/swiftwork</workdirectory>
-
- </pool>
-
-</config>
Copied: demo/cray/swift-1.0/part08-cray-loop/raven.xml (from rev 6480, demo/cray/swift-1.0/part08-cray-loop/ravenps.xml)
===================================================================
--- demo/cray/swift-1.0/part08-cray-loop/raven.xml (rev 0)
+++ demo/cray/swift-1.0/part08-cray-loop/raven.xml 2013-04-18 18:24:07 UTC (rev 6481)
@@ -0,0 +1,35 @@
+<config>
+
+ <pool handle="localhost">
+ <execution provider="local"/>
+ <filesystem provider="local"/>
+ <workdirectory>{env.PWD}/swiftwork</workdirectory>
+ </pool>
+
+ <pool handle="cray">
+ <execution provider="coaster" jobmanager="local:pbs"/>
+ <!-- <profile namespace="globus" key="project">CI-CCR000013</profile> -->
+
+ <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=32</profile>
+ <!-- <profile namespace="globus" key="queue">scalability</profile> -->
+ <!-- <profile namespace="globus" key="queue">batch</profile> -->
+ <profile namespace="globus" key="queue">small</profile>
+
+ <profile namespace="globus" key="jobsPerNode">32</profile>
+ <profile namespace="globus" key="maxTime">1800</profile>
+
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="globus" key="maxNodes">1</profile>
+
+ <profile namespace="karajan" key="jobThrottle">6.00</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+
+ <profile namespace="env" key="PATHPREFIX">{env.PATHPREFIX}</profile>
+
+ <filesystem provider="local"/>
+ <workdirectory>/dev/shm/{env.USER}/swiftwork</workdirectory>
+
+ </pool>
+
+</config>
Deleted: demo/cray/swift-1.0/part08-cray-loop/ravenps.xml
===================================================================
--- demo/cray/swift-1.0/part08-cray-loop/ravenps.xml 2013-04-18 18:12:35 UTC (rev 6480)
+++ demo/cray/swift-1.0/part08-cray-loop/ravenps.xml 2013-04-18 18:24:07 UTC (rev 6481)
@@ -1,35 +0,0 @@
-<config>
-
- <pool handle="localhost">
- <execution provider="local"/>
- <filesystem provider="local"/>
- <workdirectory>{env.PWD}/swiftwork</workdirectory>
- </pool>
-
- <pool handle="cray">
- <execution provider="coaster" jobmanager="local:pbs"/>
- <!-- <profile namespace="globus" key="project">CI-CCR000013</profile> -->
-
- <profile namespace="globus" key="providerAttributes">pbs.aprun;pbs.mpp;depth=32</profile>
- <!-- <profile namespace="globus" key="queue">scalability</profile> -->
- <!-- <profile namespace="globus" key="queue">batch</profile> -->
- <profile namespace="globus" key="queue">small</profile>
-
- <profile namespace="globus" key="jobsPerNode">32</profile>
- <profile namespace="globus" key="maxTime">1800</profile>
-
- <profile namespace="globus" key="slots">1</profile>
- <profile namespace="globus" key="nodeGranularity">1</profile>
- <profile namespace="globus" key="maxNodes">1</profile>
-
- <profile namespace="karajan" key="jobThrottle">6.00</profile>
- <profile namespace="karajan" key="initialScore">10000</profile>
-
- <profile namespace="env" key="PATHPREFIX">{env.PATHPREFIX}</profile>
-
- <filesystem provider="local"/>
- <workdirectory>/dev/shm/{env.USER}/swiftwork</workdirectory>
-
- </pool>
-
-</config>
More information about the Swift-commit
mailing list