[Swift-commit] r5463 - branches/release-0.93/docs/siteguide
davidk at ci.uchicago.edu
davidk at ci.uchicago.edu
Fri Dec 23 21:32:13 CST 2011
Author: davidk
Date: 2011-12-23 21:32:13 -0600 (Fri, 23 Dec 2011)
New Revision: 5463
Modified:
branches/release-0.93/docs/siteguide/pads
Log:
Reorganized pads documentation and give a better description of what pads is
Modified: branches/release-0.93/docs/siteguide/pads
===================================================================
--- branches/release-0.93/docs/siteguide/pads 2011-12-23 17:51:41 UTC (rev 5462)
+++ branches/release-0.93/docs/siteguide/pads 2011-12-24 03:32:13 UTC (rev 5463)
@@ -1,17 +1,41 @@
PADS
----
+PADS is a petabyte-scale, data intense computing resource located
+at the joint Argonne National Laboratory/University of Chicago
+Computation Institute. More information about PADS can be found
+at http://pads.ci.uchicago.edu.
-PADS Quickstart
-~~~~~~~~~~~~~~~
-This section will walk you through running a simple Swift script
-on PADS. It will explain how to setup software packages, how
-to create the required Swift configuration files, and finally
-how to run Swift.
+Requesting Access
+~~~~~~~~~~~~~~~~~
+If you do not already have a Computation Institute account, you can request
+access at https://www.ci.uchicago.edu/accounts. This page will give you a list
+of resources you can request access to. Be sure that PADS is selected. If
+you already have an existing CI account, but do not have access to PADS,
+send an email to support at ci.uchicago.edu to request access.
+SSH Keys
+~~~~~~~~
+Before accessing PADS, be sure to have your SSH keys configured correctly.
+There is some basic information about SSH and how to generate your key at
+http://www.ci.uchicago.edu/wiki/bin/view/Resources/SshKeys. Once you have
+followed those instructions, you can add your key at
+https://www.ci.uchicago.edu/support/sshkeys/.
+
+Connecting to a login node
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+Once your keys are configured, you should be able to access a PADS login
+node with the following command:
+
+-----
+ssh yourusername at login.pads.ci.uchicago.edu
+-----
+
Adding Software Packages
-^^^^^^^^^^^^^^^^^^^^^^^^
-In your home directory, edit a file called .soft and add these
-lines (in this order):
+~~~~~~~~~~~~~~~~~~~~~~~~
+Softenv is a system used for managing applications. In order to run Swift,
+the softenv environment will have to be modified slightly. Softenv is
+configured by a file in your home directory called .soft. Edit this file
+to look like this:
-----
+java-sun
+maui
@@ -21,6 +45,37 @@
Log out of PADS, and log back in for these changes to take effect.
+Which project(s) are you a member of?
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+PADS requires that you are a member of a project. You can determine this by
+running the following command:
+
+-----
+$ projects --available
+
+The following projects are available for your use
+
+Project PI Title
+CI-CCR000013 Michael Wilde The Swift Parallel Scripting System
+-----
+
+If you are not a member of a project, you must first request access
+to a project at http://www.ci.uchicago.edu/hpc/projects.
+
+You should make sure that you have a project set as default. Run
+the projects command with no arguments to determine if you have a default.
+
+------
+$ projects
+You have no default project set.
+-----
+
+To set your default project, use projects --set
+------
+$ projects --set CI-CCR000013 --all
+Your default project for all CI clusters has been set to CI-CCR000013.
+-----
+
Creating sites.xml
^^^^^^^^^^^^^^^^^^
Swift relies on various configuration files to determine how to
@@ -57,7 +112,6 @@
Copy a Swift Script
^^^^^^^^^^^^^^^^^^^
-
Within the Swift directory is an examples directory which contains
several introductory Swift scripts. The example we will use in this
section is called catsn.swift. Copy this script to the same directory
@@ -69,6 +123,7 @@
-----
TIP: The location of your swift directory may vary depending on how you installed it. Change this to the examples/misc directory of your installation as needed.
+
Run Swift
^^^^^^^^^
@@ -77,99 +132,21 @@
$ swift -sites.file sites.xml -tc.file tc.data catsn.swift
-----
-You should see 10 new text files get created, named catsn*.out. If
-you see these files, then you have succesfully run Swift on PADS!
+You should see several new files being created, called catsn.0001.out, catsn.0002.out, etc. Each of these
+files should contain the contents of what you placed into data.txt. If this happens, your job has run
+successfully on PADS!
TIP: Make sure your default project is defined. Read on for more information.
Read on for more detailed information about running Swift on PADS.
-Requesting Access
-~~~~~~~~~~~~~~~~~
-If you do not already have a Computation Institute account, you can request
-one at https://www.ci.uchicago.edu/accounts. This page will give you a list
-of resources you can request access to. Be sure that PADS is selected. If
-you already have an existing CI account, but do not have access to PADS,
-send an email to support at ci.uchicago.edu to request access.
-SSH Keys
-~~~~~~~~
-Before accessing PADS, be sure to have your SSH keys configured correctly.
-There is some basic information about SSH and how to generate your key at
-http://www.ci.uchicago.edu/wiki/bin/view/Resources/SshKeys. Once you have
-followed those instructions, you can add your key at
-https://www.ci.uchicago.edu/support/sshkeys/.
+Queues
+^^^^^^
-Connecting to a login node
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-Once your keys are configured, you should be able to access a PADS login
-node with the following command:
+As you run more application in the future, you will likely need
+to change queues.
------
-ssh yourusername at login.pads.ci.uchicago.edu
------
-
-Configuring softenv
-~~~~~~~~~~~~~~~~~~~
-Softenv is a system used for managing applications. In order to run Swift,
-the softenv environment will have to be modified slightly. Softenv is
-configured by a file in your home directory called .soft. Edit this file
-to look like this:
------
-+java-sun
-+maui
-+torque
- at default
------
-
-Swift Work Directory
-^^^^^^^^^^^^^^^^^^^^
-The Swift work directory is a directory which Swift uses for processing work.
-This directory needs to be writable and available to all worker nodes on
-a shared filesystem.
-
-.Examples
------
-/home/username/swiftwork
-/home/username/work
-/autonfs/gpfs-pads/projects/CI-CCR000013/myusername
------
-
-Which project(s) are you a member of?
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-PADS requires that you are a member of a project. You can determine this by
-running the following command:
-
------
-$ projects --available
-
-The following projects are available for your use
-
-Project PI Title
-
-CI-CCR000013 Michael Wilde The Swift Parallel Scripting System
------
-
-If you are not a member of a project, you must first request access
-to a project at http://www.ci.uchicago.edu/hpc/projects.
-
-You should make sure that you have a project set as default. Run
-the projects command with no arguments to determine if you have a default.
-
-------
-$ projects
-You have no default project set.
------
-
-To set your default project, use projects --set
-------
-$ projects --set CI-CCR000013 --all
-Your default project for all CI clusters has been set to CI-CCR000013.
------
-
-Determine your Queue
-^^^^^^^^^^^^^^^^^^^^
-
PADS has several different queues you can submit jobs to depending on
the type of work you will be doing. The command "qstat -q" will print
the most up to date list of this information.
@@ -185,59 +162,13 @@
|long |-- |-- |24:00:00|-- |232|130|-- | E R
|=========================================================
-Generating Configuration Files
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Now that you know what queue to use, your project, and your work directory, it is time to
-set up Swift. Swift uses a configuration file called sites.xml to determine how it should run.
-There are two methods you can use for creating this file. You can manually edit
-the configuration file, or generate it with a utility called gensites.
+When you determine your computing requirements, modify this line in your
+sites.xml:
-Manually Editing sites.xml
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-Below is the template that is used by Swift's test suite for running on PADS.
-
-----
-include::../../tests/providers/PADS/coasters/sites.template.xml[]
+<profile key="queue" namespace="globus">fast</profile>
-----
-Before you can use this configuration file, you will need to modify \_WORK_ to a directory writable by you,
-and have your default project defined. Copy this template, replace \_WORK_, and then save as sites.xml.
-
-If you wanted to use a different queue, replace "fast" with the desired queue name.
-
-Manually Editing tc.data
-~~~~~~~~~~~~~~~~~~~~~~~~
-Below is the tc.data file used by Swift's test suite for running on PADS.
-
------
-include::../../tests/providers/PADS/coasters/tc.template.data[]
-------
-
-Copy these commands and save it as tc.data.
-
-Catsn.swift
-~~~~~~~~~~~
-The swift script we will run is called catsn.swift. It simply cats a file and saves the result. This is
-a nice simple test to ensure jobs are running correctly. Create a file called data.txt which contains
-some simple input - a "hello world" will do the trick.
-
------
-include::../../examples/misc/catsn.swift[]
------
-
-
-Running Swift
-~~~~~~~~~~~~~
-Now that everything is in place, run Swift with the following command:
-
------
-swift -sites.file sites.xml -tc.file tc.data catsn.swift -n=10
------
-
-You should see several new files being created, called catsn.0001.out, catsn.0002.out, etc. Each of these
-files should contain the contents of what you placed into data.txt. If this happens, your job has run
-successfully on PADS!
-
More Help
~~~~~~~~~
The best place for additional help is the Swift user mailing list. You can subscribe to this list at
More information about the Swift-commit
mailing list