[Swift-commit] r7301 - trunk/docs/siteguide
davidk at ci.uchicago.edu
davidk at ci.uchicago.edu
Thu Nov 21 16:54:30 CST 2013
Author: davidk
Date: 2013-11-21 16:54:29 -0600 (Thu, 21 Nov 2013)
New Revision: 7301
Modified:
trunk/docs/siteguide/beagle
trunk/docs/siteguide/fusion
trunk/docs/siteguide/grid
trunk/docs/siteguide/intrepid
trunk/docs/siteguide/siteguide.txt
trunk/docs/siteguide/stampede
trunk/docs/siteguide/uc3
Log:
Point older osg sites to newer tutorial script
Remove references to www.ci.uchicago.edu/swift
Modified: trunk/docs/siteguide/beagle
===================================================================
--- trunk/docs/siteguide/beagle 2013-11-21 22:03:55 UTC (rev 7300)
+++ trunk/docs/siteguide/beagle 2013-11-21 22:54:29 UTC (rev 7301)
@@ -14,7 +14,7 @@
workings can be found on Swift documentation page here:
----
-http://www.ci.uchicago.edu/swift/wwwdev/docs/index.php
+http://swiftlang.org/docs/index.php
----
More information on Beagle can be found on UChicago Beagle website here:
@@ -119,8 +119,7 @@
pbs cat /bin/cat null null null
-----
-More about config and tc file options can be found in the swift userguide here:
-http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html#_swift_configuration_properties
+More about config and tc file options can be found in the Swift link:http://swiftlang.org/docs/index.php[documentation].
*step 6.* Run the example using following commandline:
Modified: trunk/docs/siteguide/fusion
===================================================================
--- trunk/docs/siteguide/fusion 2013-11-21 22:03:55 UTC (rev 7300)
+++ trunk/docs/siteguide/fusion 2013-11-21 22:54:29 UTC (rev 7301)
@@ -51,7 +51,7 @@
you can copy and paste to get running quickly. The sites.xml file
tells Swift how to submit jobs, where working directories are
located, and various other configuration information. More
-information on sites.xml can be found in the Swift link:http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html[user guide].
+information on sites.xml can be found in the Swift link:http://swiftlang.org/docs/index.php[documentation].
The first step is to paste the text below into a file named sites.xml.
Modified: trunk/docs/siteguide/grid
===================================================================
--- trunk/docs/siteguide/grid 2013-11-21 22:03:55 UTC (rev 7300)
+++ trunk/docs/siteguide/grid 2013-11-21 22:54:29 UTC (rev 7301)
@@ -1,339 +1,4 @@
Grids: Open Science Grid and TeraGrid
-------------------------------------
-
-Overview of running on grid sites
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-* Get a DOEGrids cert. Then register it in the *OSG Engage VO*, and/or
-map it using +gx-request+ on TeraGrid sites.
-
-* Run +GridSetup+ to configure Swift to use the grid sites. This tests
-for correct operation and creates a "green list" of good sites.
-
-* Prepare an installation package for the programs you want to run on
-grid sites via Swift, and install that package using +foreachsite+.
-
-* Run +swift-workers+ to start and maintain a pool of Swift workers on
-each site.
-
-* Run Swift scripts that use the grid site resources.
-
-NOTE: This revision only supports a single-entry sites file which uses
-provider staging and assumes that the necessary apps are locatable
-through the same tc entries (ie either absolute or PATH-relative
-paths) on all sites.
-
-NOTE: This revision has been testing using the bin/grid code from
-trunk (which gets installed into trunk's bin/ directory, and the base
-swift code from the 0.93 branch. No other configurations have been
-tested at the moment. I intend to put this code in bin/grid in 0.93,
-as it should have no ill affects on other Swift usage.
-
-Requesting Access
-~~~~~~~~~~~~~~~~~
-
-*For OSG*: Obtain a DOEGrids certificate and register the certificate in
-the OSG "Engage" VO following the procedure at:
-
-https://twiki.grid.iu.edu/bin/view/Engagement/EngageNewUserGuide
-
-[red]*FIXME*: access to OSG wiki pages may request the user to present
-a certificate. Is this a problem from users without one? If so, make a
-copy of the page on the Swift web.
-
-*For TeraGrid*: Obtain a DOEGrids certifcate using the OSG Engage
-instructions above. Ask a TeraGrid PI to add you to a TeraGrid
-project. Once you obtain a login and project access (via US Mail), use
-gx-request to add your certificate.
-
-A detailed step-by-step instructions for requesting and installing your
-certificates in the browser and client machine are as follows:
-
-*Step1.* Apply for a certificate: https://pki1.doegrids.org/ca/; use ANL as
-affiliation (registration authority) in the form.
-
-*Step2.* When you receive your certificate via a link by mail, download and
-install it in your browser; we have tested it for firefox on linux and mac.,
-and for Chrome on mac.
-
-On firefox, as you click the link that you received in the mail, you will be
-prompted to install it by firefox: passphrase it and click install. Next take a
-backup of this certificate in the form of .p12.
-This is in Preferences > Advanced > Encryption > View Certificate > Your
-Certificate
-
-*Step3.* Install DOE CA and ESnet root CA into your browser by clicking the top
-left links on this page: http://www.doegrids.org/
-
-*Step4.* Go to the Engage VO registration point here:
-https://osg-engage.renci.org:8443/vomrs/Engage/vomrs from the same browser that
-has the above certs installed. Also see
-https://twiki.grid.iu.edu/bin/view/Engagement/EngageNewUserGuide for more
-details.
-
-*Step5.* For installation of certificate on client machine, you need to have the certificate
-that is in the browser put in your client's ~/.globus directory from where you want to access OSG
-resources. The certificate has to be in the form of .pem files with a seperate
-.pem file for key and cert. For this conversion use the above backed up .p12 file as follows:
-
-----
-$ openssl pkcs12 -in your.p12 -out usercert.pem -nodes -clcerts -nokeys
-$ openssl pkcs12 -in your.p12 -out userkey.pem -nodes -nocerts
-----
-
-Above commands are taken from:
-http://security.ncsa.illinois.edu/research/grid-howtos/usefulopenssl.html
-For more on openssl: http://www.openssl.org/docs/apps/openssl.html
-
-Step6. Test it:
-
-----
-$ voms-proxy-init --voms Engage -hours 48
-----
-
-To run jobs using the procedure described here, you need to be logged
-in to a "submit host" on which you will run Swift and other
-grid-related utilities. A submit host is any host where the OSG client stack
-or equivalent tools are installed. Such hosts include the OSG Engage
-submit host (engage-submit3.renci.org), and the two Swift lab servers
-{bridled,communicado}.ci.uchicago.edu.
-
-Obtain a login on engage-submit3.renci.org following instructions on
-the OSG URL above.
-
-Obtain a CI login with access to the Swift lab servers by requesting
-"OSG Gridlab" access at:
-
-http://accounts.ci.uchicago.edu
-
-
-Connecting to a submit host
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
------
-ssh yourusername at bridled.ci.uchicago.edu
-ssh yourusername at communicado.ci.uchicago.edu
-ssh yourusername at engage-submit.renci.org
------
-
-Downloading and install Swift
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The current version of Swift can be downloaded from
-http://www.ci.uchicago.edu/swift/downloads/index.php.
-
-Fetch and untar the latest release.
-Then add the Swift bin/ directory to your PATH. For example:
-
------
-cd $HOME
-wget http://www.ci.uchicago.edu/swift/packages/swift-0.92.1.tar.gz
-tar txf swift-0.92.1.tar.gz
-export PATH=$PATH:$HOME/swift-0.92.1/bin
------
-
-Set up OSG environment
-~~~~~~~~~~~~~~~~~~~~~~
-
-Depending on your shell type, run:
-
------
-source /opt/osg-<version>/setup.sh
-or
-source /opt/osg-<version>/setup.csh
------
-
-NOTE: This above step is not required on engage-submit3 host.
-
-Create a VOMS Grid proxy
-~~~~~~~~~~~~~~~~~~~~~~~~
-
------
-$ voms-proxy-init -voms Engage -valid 72:00
------
-
-Generating Configuration Files
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
------
-cd $HOME
-mkdir swiftgrid
-cd swiftgrid
-gen_gridsites
-# Wait a few minutes to a few hours for Swift to validate grid sites
-get_greensites >greensites
------
-
-You can repeatedly try the get_greensites command, which simply
-concatenates all the site names that sucessfully resturned an output
-file from site tests.
-
-Installing software on OSG sites
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The command "foreachsite" will execute a local shell script passed to
-it as an argument, on each OSG site in the Engage VO ReSS site
-catalog. The user's installscript will execute on either the head node
-(as a GRAM "fork" job) or on a worker node, as controlled by the
--resource option. Its syntax is:
-
------
-$ ./foreachsite -help
-./foreachsite [-resource fork|worker ] [-sites alt-sites-file] scriptname
-$
------
-
-To install your software, create a script similar to "myapp.sh",
-below, which (in this example) reads a tar file of a pre-compiled
-application "myapp-2.12" and executes a test script for that
-application. The test script should print some reconizable indication
-of its success or failure:
-
------
-$ cat myinstall.sh
-renice 2 -p $$
-IDIR=OSG_APP/extenci/myname/myapp
-mkdir -p $IDIR
-cd $IDIR
-wget http://my.url.edu/~mydir/myapp-2.12.tar.tgz
-tar zxf myapp-2.12.tar.tgz
-myapp-2.12/bin/test_myapp.sh
-if [ $? = 0 ]; then
- echo INSTALL SUCCEEDED
-else
- echo INSTALL FAILED
-fi
-$
-$ foreachsite -resource fork myinstall.sh
-$
-$ # Wait a while here, then poll for successfully installed apps...
-$
-$ grep SUCCEEDED run.89/*/*.stdout
-$
------
-
-Following is an example of the installation script for DSSAT app on OSG:
-
------
-#!/bin/bash
-
-cd ${OSG_APP}/extenci/swift/
-
-#pull
-wget http://www.ci.uchicago.edu/~ketan/DSSAT.tgz
-
-#extract
-tar zxf DSSAT.tgz
-
-# test
-cd DSSAT/data
-
-../DSSAT040.EXE A H1234567.MZX > std.OUT
-
-if [ $? = 0 ]; then
- echo INSTALL SUCCEEDED
-else
- echo INSTALL FAILED
-fi
------
-
-Starting a single coaster service
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-This single coaster service will service all grid sites:
-
------
-start-grid-service --loglevel INFO --throttle 3.99 --jobspernode 1 \
- >& start-grid-service.out
------
-
-Starting workers on OSG sites through GRAM
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Make sure that your "greensites" file is in the current working
-directory.
-
-The swiftDemand file should be set to contain the number of workers
-you want to start across all OSG sites. Eventually this will be set
-dynamically by watching your Swift script execute. (Note that this
-number only includes jobs started by the swift-workers factory
-command, not by any workers added manually from the TeraGrid - see
-below.
-
-The condor directory must be pre-created and will be used by Condor to
-return stdout and stderr files from the Condor jobs, which will
-execute the wrapper script "run-workers.sh".
-
-NOTE: this script is current built manually, and wraps around and
-transports the worker.pl script. This needs to be automated.
-
------
-echo 250 >swiftDemand mkdir -p condor
-
-swift-workers greensites extenci \
- http://communicado.ci.uchicago.edu:$(cat service-0.wport) \
- >& swift-workers.out &
------
-
-Starting workers on OSG sites through GlideinWMS
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-As an alternative to the above gram based 'direct' worker submission, a GlideinWMS based worker submission can be made. The service start step would be same as above.
-
-GlideinWMS is a Glidein Based WMS (Workload Management System) that works on top of condor.
-
-As with the case of Gram based workers, the condor directory must be pre-created and will be used by Condor to
-return stdout and stderr files from the Condor jobs, which will
-execute the wrapper script "run-workers.sh".
-
------
-run-gwms-workers http://communicado.ci.uchicago.edu:$(cat service-0.wport) \
-100 >& gwms-workers.out &
------
-
-NOTE: The run-gwms-workers is available from the bin/grid directory of swift trunk code. You will need to include it in your PATH.
-
-In the above commandline, one can change the number of workers by changing the second commandline argument, which is 100 in this example.
-
-Adding workers from TeraGrid sites
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The job below can be used to submit jobs to TareGrid (Ranger only at
-the moment) to add more workers to the execution pool. The same
-requirements hold there as for OSG sites, namely, that the app tools
-listed in tc for the single execution site need to be locatable on the
-TeraGrid site(s).
-
------
-start-ranger-service --nodes 1 --walltime 00:10:00 --project TG-DBS123456N \
- --queue development --user tg12345 --startservice no \
- >& start-ranger-service.out
------
-
-NOTE: Change the project and user names to match your TeraGrid
-parameters.
-
-Running Swift
-~~~~~~~~~~~~~
-Now that everything is in place, run Swift with the following command:
-
------
-export SWIFT_HEAP_MAX=6000m # Add this for very large scripts
-
-swift -config cf.ps -tc.file tc -sites.file sites.grid-ps.xml \
- catsn.swift -n=10000 >& swift.out &
------
-
-You should see several new files being created, called catsn.0001.out,
-catsn.0002.out, etc. Each of these files should contain the contents
-of what you placed into data.txt. If this happens, your job has run
-successfully on the grid sites.
-
-More Help
-~~~~~~~~~
-
-The best place for additional help is the Swift user mailing list. You
-can subscribe to this list at
-http://mail.ci.uchicago.edu/mailman/listinfo/swift-user. When
-submitting information, please send your sites.xml file, your tc.data,
-and any Swift log files that were created during your attempt.
+Please refer to the
+link:../../../tutorials/osgconnect/tutorial.html[Swift Tutorial for OSGConnect].
Modified: trunk/docs/siteguide/intrepid
===================================================================
--- trunk/docs/siteguide/intrepid 2013-11-21 22:03:55 UTC (rev 7300)
+++ trunk/docs/siteguide/intrepid 2013-11-21 22:54:29 UTC (rev 7301)
@@ -41,7 +41,7 @@
Downloading and building Swift
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The most recent versions of Swift can be found at
-http://www.ci.uchicago.edu/swift/downloads/index.php. Follow the instructions
+http://swiftlang.org/downloads/index.php. Follow the instructions
provided on that site to download and build Swift.
Adding Swift to your PATH
Modified: trunk/docs/siteguide/siteguide.txt
===================================================================
--- trunk/docs/siteguide/siteguide.txt 2013-11-21 22:03:55 UTC (rev 7300)
+++ trunk/docs/siteguide/siteguide.txt 2013-11-21 22:54:29 UTC (rev 7301)
@@ -37,5 +37,3 @@
include::stampede[]
include::uc3[]
-
-link:http://www.ci.uchicago.edu/swift/docs/index.php[home]
Modified: trunk/docs/siteguide/stampede
===================================================================
--- trunk/docs/siteguide/stampede 2013-11-21 22:03:55 UTC (rev 7300)
+++ trunk/docs/siteguide/stampede 2013-11-21 22:54:29 UTC (rev 7301)
@@ -9,7 +9,7 @@
Downloading and building Swift
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The most recent versions of Swift can be found at
-http://www.ci.uchicago.edu/swift/downloads/index.php[the Swift downloads page]. Follow the instructions
+http://swiftlang.org/downloads/index.php[the Swift downloads page]. Follow the instructions
provided on that site to download and build Swift.
Overview of How to Run
Modified: trunk/docs/siteguide/uc3
===================================================================
--- trunk/docs/siteguide/uc3 2013-11-21 22:03:55 UTC (rev 7300)
+++ trunk/docs/siteguide/uc3 2013-11-21 22:54:29 UTC (rev 7301)
@@ -1,270 +1,5 @@
-UC3 (x86 cluster)
-----------------
+UC3 / OSGConnect (x86 cluster)
+------------------------------
+Please refer to the
+link:../../../tutorials/osgconnect/tutorial.html[Swift Tutorial for OSGConnect].
-Requesting Access
-~~~~~~~~~~~~~~~~~
-To request access to UC3, you must have a University of Chicago CNetID
-and be a meimber of the UC3 group. More information about UC3 can be
-found at https://wiki.uchicago.edu/display/uc3/UC3+Home or
-uc3-support at lists.uchicago.edu.
-
-Connecting to a login node
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-To access the UC3 login node, you will use your CNetID and password.
-
------
-ssh -l <cnetid> uc3-sub.uchicago.edu
------
-
-Installing Swift
-~~~~~~~~~~~~~~~~
-Swift should be available by default on the UC3 login nodes. You can verify
-this by running the following command:
------
-swift -version
------
-
-If for some reason Swift is not available, you can following the instructions at
-http://www.ci.uchicago.edu/swift/guides/release-0.93/quickstart/quickstart.html.
-Swift 0.94 or later is required to work with the condor provider on UC3.
-
-Creating sites.xml
-~~~~~~~~~~~~~~~~~~
-This section will provide a working configuration file which you can copy and paste
-to get running quickly. The sites.xml file tells Swift how to submit jobs, where
-working directories are located, and various other configuration information.
-More information on sites.xml can be found in the Swift User’s Guide.
-
-The first step is to paste the text below into a file named sites.xml:
------
-<config>
- <pool handle="uc3">
- <execution provider="coaster" url="uc3-sub.uchicago.edu" jobmanager="local:condor"/>
- <profile namespace="karajan" key="jobThrottle">999.99</profile>
- <profile namespace="karajan" key="initialScore">10000</profile>
- <profile namespace="globus" key="jobsPerNode">1</profile>
- <profile namespace="globus" key="maxWalltime">3600</profile>
- <profile namespace="globus" key="nodeGranularity">1</profile>
- <profile namespace="globus" key="highOverAllocation">100</profile>
- <profile namespace="globus" key="lowOverAllocation">100</profile>
- <profile namespace="globus" key="slots">1000</profile>
- <profile namespace="globus" key="maxNodes">1</profile>
- <profile namespace="globus" key="condor.+AccountingGroup">"group_friends.{env.USER}"</profile>
- <profile namespace="globus" key="jobType">nonshared</profile>
- <filesystem provider="local" url="none" />
- <workdirectory>.</workdirectory>
- </pool>
-</config>
------
-
-Creating tc.data
-~~~~~~~~~~~~~~~~
-The tc.data configuration file gives information about the applications that will be called by Swift.
-More information about the format of tc.data can be found in the Swift User’s guide.
-
-Paste the following example into a file named tc.data:
------
-uc3 echo /bin/echo null null null
------
-
-Create a configuration file
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-A swift configuration file enables and disables some settings in Swift. More information on what
-these settings do can be found in the Swift User's guide.
-
-Paste the following lines into a file called cf:
------
-wrapperlog.always.transfer=false
-sitedir.keep=true
-execution.retries=0
-lazy.errors=false
-status.mode=provider
-use.provider.staging=true
-provider.staging.pin.swiftfiles=false
-use.wrapper.staging=false
------
-
-Creating echo.swift
-~~~~~~~~~~~~~~~~~~~
-Now we need to create a swift script to test with. Let's use a simple application that calls /bin/echo.
-
------
-type file;
-
-app (file o) echo (string s) {
- echo s stdout=@o;
-}
-
-foreach i in [1:5] {
- file output_file <single_file_mapper; file=@strcat("output/output.", i, ".txt")>;
- output_file = echo( @strcat("This is test number ", i) );
-}
------
-
-Running Swift
-~~~~~~~~~~~~~
-Putting everything together now, run your Swift script with the following command:
-
------
-swift -sites.file sites.xml -tc.file tc.data -config cf echo.swift
------
-
-If everything runs successfully, you will see 5 files get created in the output directory.
-
-
-Controlling where jobs run
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-Swift will automatically generate condor scripts for you with the basic information about
-how to run. However, condor has hundreds of commands that let you customize how things work.
-If you need one of these advanced commands, you can add it to your sites.xml. The basic
-template for this is:
-
------
-<profile namespace="globus" key="condor.key">value</profile>
------
-
-For example, let's assume that you want to control where your jobs run by adding a
-requirement. The condor command that will control the run is:
------
-Requirements = UidDomain == "osg-gk.mwt2.org"
------
-
-To have this generated by Swift, you will add a line to your sites.xml in the key/value style
-shown above.
------
-<profile namespace="globus" key="condor.Requirements">UidDomain == "osg-gk.mwt2.org"</profile>
------
-
-Installing Application Scripts on HDFS
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-NOTE: This section will only work if the application you want to use is an interpreted script (bash,
-python, perl, etc). HDFS does not have the ability to give programs an execution bit and run them. If
-the application you want to run on UC3 is a compiled executable, skip this section and read ahead.
-
-Once your simple echo test is running, you'll want to start using your own applications. One way
-to go about doing this is by using the Hadoop filesystem. This filesystem is only available on
-the UC3 Seeder Cluster. In order to limit yourself to machines that can access this filesystem,
-add the following line to your sites.xml file:
-
------
-<profile namespace="globus" key="condor.Requirements">UidDomain == "osg-gk.mwt2.org" && regexp("uc3-c*", Machine)</profile>
------
-
-Now you can install your script somewhere under the directory /mnt/hadoop/users/<yourusername>.
-Here is an example with putting everything together.
-
-.sites.xml
------
-<config>
- <pool handle="uc3">
- <execution provider="coaster" url="uc3-sub.uchicago.edu" jobmanager="local:condor"/>
- <profile namespace="karajan" key="jobThrottle">999.99</profile>
- <profile namespace="karajan" key="initialScore">10000</profile>
- <profile namespace="globus" key="jobsPerNode">1</profile>
- <profile namespace="globus" key="maxWalltime">3600</profile>
- <profile namespace="globus" key="nodeGranularity">1</profile>
- <profile namespace="globus" key="highOverAllocation">100</profile>
- <profile namespace="globus" key="lowOverAllocation">100</profile>
- <profile namespace="globus" key="slots">1000</profile>
- <profile namespace="globus" key="maxNodes">1</profile>
- <profile namespace="globus" key="condor.+AccountingGroup">"group_friends.{env.USER}"</profile>
- <profile namespace="globus" key="jobType">nonshared</profile>
- <profile namespace="globus" key="condor.Requirements">UidDomain == "osg-gk.mwt2.org" && regexp("uc3-c*", Machine)</profile>
- <filesystem provider="local" url="none" />
- <workdirectory>.</workdirectory>
- </pool>
-</config>
------
-
-.tc.data
------
-uc3 bash /bin/bash null null null
------
-
-.myscript.swift
------
-type file;
-
-app (file o) myscript ()
-{
- bash "/mnt/hadoop/users/<yourusername>/myscript.sh" stdout=@o;
-}
-
-file out[]<simple_mapper; location="outdir", prefix="myscript.",suffix=".out">;
-int ntasks = @toInt(@arg("n","1"));
-
-foreach n in [1:ntasks] {
- out[n] = myscript();
-}
------
-
-./mnt/hadoop/users/<yourusername>/myscript.sh
------
-#!/bin/bash
-
-echo This is my script
------
-
-.cf
------
-wrapperlog.always.transfer=false
-sitedir.keep=true
-execution.retries=0
-lazy.errors=false
-status.mode=provider
-use.provider.staging=true
-provider.staging.pin.swiftfiles=false
-use.wrapper.staging=false
------
-
-.Example run
------
-$ swift -sites.file sites.xml -tc.file tc.data -config cf myscript.swift -n=10
-Swift trunk swift-r6146 cog-r3544
-
-RunID: 20130109-1657-tf01jpaa
-Progress: time: Wed, 09 Jan 2013 16:58:00 -0600
-Progress: time: Wed, 09 Jan 2013 16:58:30 -0600 Submitted:10
-Progress: time: Wed, 09 Jan 2013 16:59:00 -0600 Submitted:10
-Progress: time: Wed, 09 Jan 2013 16:59:12 -0600 Stage in:1 Submitted:9
-Final status: Wed, 09 Jan 2013 16:59:12 -0600 Finished successfully:10
-$ ls outdir/*
-outdir/myscript.0001.out outdir/myscript.0003.out outdir/myscript.0005.out outdir/myscript.0007.out outdir/myscript.0009.out
-outdir/myscript.0002.out outdir/myscript.0004.out outdir/myscript.0006.out outdir/myscript.0008.out outdir/myscript.0010.out
------
-
-Staging in Applications with Coaster Provider Staging
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-If you want your application to be as portable as possible, you can use
-coaster provider staging to send your application(s) to a remote node.
-By removing the condor requirements in the previous section, you will
-have more cores available. Here is a simple script that stages in and
-executes a shell script.
-
------
-type file;
-
-app (file o) sleep (file script, int delay)
-{
- # chmod +x script.sh ; ./script.sh delay
- bash "-c" @strcat("chmod +x ./", @script, " ; ./", @script, " ", delay) stdout=@o;
-}
-
-file sleep_script <"sleep.sh">;
-
-foreach i in [1:5] {
- file o <single_file_mapper; file=@strcat("output/output.", i, ".txt")>;
- o = sleep(sleep_script, 10);
-}
------
-
-Mapping our script to a file and passing it as an argument to an app function
-causes the application to be staged in. The only thing we need on the worker
-node is /bin/bash.
-
-TIP: If the program you are staging in is an executable, statically compiling it
-will increase the chances of a successful run.
-
-TIP: If the application you are staging is more complex, with multiple files,
-add everything you need into just one compressed tar file, then extract on the
-worker node.
More information about the Swift-commit
mailing list