[Swift-commit] r7856 - branches/release-0.94/docs/siteguide
davidk at ci.uchicago.edu
davidk at ci.uchicago.edu
Mon May 12 11:01:47 CDT 2014
Author: davidk
Date: 2014-05-12 11:01:47 -0500 (Mon, 12 May 2014)
New Revision: 7856
Added:
branches/release-0.94/docs/siteguide/blues
Modified:
branches/release-0.94/docs/siteguide/fusion
branches/release-0.94/docs/siteguide/siteguide.txt
Log:
Add a site guide entries for blues, update fusion guide
Added: branches/release-0.94/docs/siteguide/blues
===================================================================
--- branches/release-0.94/docs/siteguide/blues (rev 0)
+++ branches/release-0.94/docs/siteguide/blues 2014-05-12 16:01:47 UTC (rev 7856)
@@ -0,0 +1,126 @@
+Blues (x86 cluster)
+-------------------
+
+Fusion is a 310-node computing cluster for the Argonne
+National Laboratory community. The primary goal of the LCRC is to
+facilitate mid-range computing in all of the scientific programs of
+Argonne and the University of Chicago.
+
+This section will walk you through running a simple Swift script
+on Fusion.
+
+Requesting Access
+~~~~~~~~~~~~~~~~~
+If you do not already have a Blues account, you can request one at
+https://accounts.lcrc.anl.gov/request.php. Email support at lcrc.anl.gov
+for additional help.
+
+Projects
+~~~~~~~~
+In order to run a job on a Blues compute node, you must first be associated
+with a project.
+
+Each project has one or more Primary Investigators, or PIs. These PIs are
+responsible for adding and removing users to a project. Contact the PI of
+your project to be added.
+
+More information on this process can be found at
+http://www.lcrc.anl.gov/info/Projects.
+
+SSH Keys
+~~~~~~~~
+Before accessing Blues, be sure to have your SSH keys configured correctly.
+SSH keys are required to access fusion. You should see information about
+this when you request your account. Email support at lcrc.anl.gov for
+additional help.
+
+Connecting to a login node
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+Once your keys are configured, you should be able to access a Fusion login
+node with the following command:
+
+-----
+ssh yourusername at blues.lcrc.anl.gov
+-----
+
+Creating sites.xml
+~~~~~~~~~~~~~~~~~~
+This section will provide a working configuration file which
+you can copy and paste to get running quickly. The sites.xml file
+tells Swift how to submit jobs, where working directories are
+located, and various other configuration information. More
+information on sites.xml can be found in the Swift User's Guide.
+
+The first step is to paste the text below into a file named sites.xml.
+
+-----
+<?xml version="1.0" encoding="UTF-8"?>
+<config xmlns="http://www.ci.uchicago.edu/swift/SwiftSites">
+ <pool handle="blues">
+ <execution jobmanager="local:pbs" provider="coaster"/>
+ <filesystem provider="local" url="none" />
+ <profile namespace="globus" key="maxtime">3600</profile>
+ <profile namespace="globus" key="jobsPerNode">16</profile>
+ <profile namespace="globus" key="ppn">16</profile>
+ <profile namespace="globus" key="queue">shared</profile>
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="nodeGranularity">1</profile>
+ <profile namespace="globus" key="maxNodes">1</profile>
+ <profile namespace="karajan" key="jobThrottle">1.00</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <workdirectory>/home/{env.USER}/swiftwork</workdirectory>
+ </pool>
+</config>
+-----
+
+Creating tc.data
+~~~~~~~~~~~~~~~~
+The tc.data configuration file gives information about the applications
+that will be called by Swift. More information about the format
+of tc.data can be found in the Swift User's guide.
+
+Paste the following example into a file named tc.data
+
+-----
+blues hostname /bin/hostname
+-----
+
+Create a Swift Script
+~~~~~~~~~~~~~~~~~~~~~
+The following script runs /bin/hostname utility 100 times and writes
+the outputs. Paste the following script into a file called hostname.swift.
+
+-----
+type file;
+
+app (file o) hostname ()
+{
+ hostname stdout=@o;
+}
+
+foreach i in [1:100] {
+ file hostname_log <single_file_mapper; file=@strcat(i, ".hostname.log")>;
+ hostname_log = hostname();
+}
+-----
+
+Run Swift
+~~~~~~~~~
+Finally, run the script
+-----
+$ swift -sites.file sites.xml -tc.file tc.data hostname.swift
+-----
+
+When this runs, you should see 100 new text files get created, named *.hostname.log. If
+you see these files, then you have succesfully run Swift on Blues!
+
+Queues
+~~~~~~
+Blues has multiple queues available. The command "qstat -q" will print a list.
+
+To edit the queue, modify the following line in sites.xml
+
+-----
+<profile namespace="globus" key="queue">batch</profile>
+-----
+
Modified: branches/release-0.94/docs/siteguide/fusion
===================================================================
--- branches/release-0.94/docs/siteguide/fusion 2014-05-12 14:43:28 UTC (rev 7855)
+++ branches/release-0.94/docs/siteguide/fusion 2014-05-12 16:01:47 UTC (rev 7856)
@@ -1,6 +1,5 @@
Fusion (x86 cluster)
-------------------
-
Fusion is a 320-node computing cluster for the Argonne
National Laboratory community. The primary goal of the LCRC is to
facilitate mid-range computing in all of the scientific programs of
@@ -55,16 +54,25 @@
The first step is to paste the text below into a file named sites.xml.
-----
-include::../../tests/providers/fusion/coasters/sites.template.xml[]
+<config>
+ <pool handle="fusion">
+ <execution jobmanager="local:pbs" provider="coaster"/>
+ <filesystem provider="local" url="none" />
+ <profile namespace="globus" key="maxtime">3600</profile>
+ <profile namespace="globus" key="jobsPerNode">8</profile>
+ <profile namespace="globus" key="slots">1</profile>
+ <profile namespace="globus" key="nodeGranularity">2</profile>
+ <profile namespace="globus" key="maxNodes">2</profile>
+ <profile namespace="globus" key="queue">shared</profile>
+ <profile namespace="karajan" key="jobThrottle">5.99</profile>
+ <profile namespace="karajan" key="initialScore">10000</profile>
+ <profile namespace="globus" key="HighOverAllocation">1000</profile>
+ <profile namespace="globus" key="LowOverAllocation">1000</profile>
+ <workdirectory>/homes/{env.USER}/swiftwork</workdirectory>
+ </pool>
+</config>
-----
-This file will require one customization. Create a
-directory called swiftwork. Modify \_WORK_ in sites.xml
-to point to this new directory. For example
------
-<workdirectory>/home/myhome/swiftwork</workdirectory>
------
-
Creating tc.data
~~~~~~~~~~~~~~~~
The tc.data configuration file gives information about the applications
@@ -74,30 +82,36 @@
Paste the following example into a file named tc.data
-----
-include::../../tests/providers/fusion/coasters/tc.template.data[]
+fusion hostname /bin/hostname
-----
-Copy a Swift Script
-~~~~~~~~~~~~~~~~~~~~
-Within the Swift directory is an examples directory which contains
-several introductory Swift scripts. The example we will use in this
-section is called catsn.swift. Copy this script to the same directory
-that your sites.xml and tc.data files are located.
+Create a Swift Script
+~~~~~~~~~~~~~~~~~~~~~
+The following script runs /bin/hostname utility 100 times and writes
+the outputs. Paste the following script into a file called hostname.swift.
-----
-$ cp ~/swift/examples/misc/catsn.swift .
-$ cp ~/swift/examples/misc/data.txt .
+type file;
+
+app (file o) hostname ()
+{
+ hostname stdout=@o;
+}
+
+foreach i in [1:100] {
+ file hostname_log <single_file_mapper; file=@strcat(i, ".hostname.log")>;
+ hostname_log = hostname();
+}
-----
-TIP: The location of your swift directory may vary depending on how you installed it. Change this to the examples/misc directory of your installation as needed.
Run Swift
~~~~~~~~~
Finally, run the script
-----
-$ swift -sites.file sites.xml -tc.file tc.data catsn.swift
+$ swift -sites.file sites.xml -tc.file tc.data hostname.swift
-----
-You should see 10 new text files get created, named catsn*.out. If
+When this runs, you should see 100 new text files get created, named *.hostname.log. If
you see these files, then you have succesfully run Swift on Fusion!
Queues
@@ -115,10 +129,3 @@
More information on Fusion queues can be found at
http://www.lcrc.anl.gov/info/BatchJobs.
-
-More Help
-~~~~~~~~~
-The best place for additional help is the Swift user mailing list. You can subscribe to this list at
-https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user. When submitting information, please send your sites.xml file, your tc.data, and any Swift log files that were created during your attempt.
-
-
Modified: branches/release-0.94/docs/siteguide/siteguide.txt
===================================================================
--- branches/release-0.94/docs/siteguide/siteguide.txt 2014-05-12 14:43:28 UTC (rev 7855)
+++ branches/release-0.94/docs/siteguide/siteguide.txt 2014-05-12 16:01:47 UTC (rev 7856)
@@ -12,6 +12,8 @@
include::beagle[]
+include::blues[]
+
include::fusion[]
include::futuregrid[]
More information about the Swift-commit
mailing list