[Swift-commit] r5259 - trunk/docs/userguide

[wozniak at ci.uchicago.edu]

Author: wozniak
Date: 2011-10-26 11:33:11 -0500 (Wed, 26 Oct 2011)
New Revision: 5259

Modified:
   trunk/docs/userguide/profiles
Log:
Add documentation about Swift dynamic profiles


Modified: trunk/docs/userguide/profiles
===================================================================
--- trunk/docs/userguide/profiles	2011-10-23 04:03:13 UTC (rev 5258)
+++ trunk/docs/userguide/profiles	2011-10-26 16:33:11 UTC (rev 5259)
@@ -1,15 +1,15 @@
-Profiles
---------
-Profiles are configuration parameters than can be specified either for
-sites or for transformation catalog entries. They influence the
-behaviour of Swift towards that site (for example, by changing the load
-Swift will place on that sites) or when running a particular procedure.
+== Profiles
 
+Profiles influence the behaviour of Swift when running an +app+ task.
+They are configuration parameters than can be specified for
+sites, for transformation catalog entries, or on a task-by-task as a dynamic
+profile.
+
 Profile entries for a site are specified in the site catalog. Profile
 entries for specific procedures are specified in the transformation
-catalog.
+catalog.  Profile entries for a given task are specified in the +app+
+definition as a dynamic profile.
 
-
 Karajan namespace
 ~~~~~~~~~~~~~~~~~
 maxSubmitRate limits the maximum rate of job submission, in jobs per
@@ -105,7 +105,7 @@
 used by the GRAM2 and GRAM4 providers.
 
 condor_requirements allows a requirements string to be specified when
-Condor is used as an LRM behind GRAM2. Example: 
+Condor is used as an LRM behind GRAM2. Example:
 
 ----
 <profile namespace="globus" key="condor_requirements">Arch == "X86_64" || Arch="INTEL"</profile>
@@ -171,7 +171,7 @@
 maxnodes - Determines the maximum number of nodes that can be allocated
 in one coaster block. Default: unlimited.
 
-maxtime - Indicates the maximum walltime, in seconds, that a coaster 
+maxtime - Indicates the maximum walltime, in seconds, that a coaster
 block can have.
 Default: unlimited.
 
@@ -179,11 +179,11 @@
 Swing window showing, graphically, the state of the coaster scheduler
 (blocks, jobs, etc.). Default: false
 
-internalhostname - If the head node has multiple network interfaces, 
-only one of which is visible from the worker nodes. The choice of 
-which interface is the one that worker nodes can connect to is a 
-matter of the particular cluster. This must be set in the your 
-sites file to clarify to the workers which exact interface on the 
+internalhostname - If the head node has multiple network interfaces,
+only one of which is visible from the worker nodes. The choice of
+which interface is the one that worker nodes can connect to is a
+matter of the particular cluster. This must be set in the your
+sites file to clarify to the workers which exact interface on the
 head node they are to try to connect to.
 
 env namespace
@@ -212,8 +212,43 @@
 about the remote site to be gathered and returned to the submit side.
 (since Swift 0.9)
 
-SWIFT_GEN_SCRIPTS - set in the env namespace profiles. This variable 
+SWIFT_GEN_SCRIPTS - set in the env namespace profiles. This variable
 just needs to be set, it doesn't matter what it is set to. If set, then Swift
 will keep the script that was used to execute the job in the job directory.
 The script will be called run.sh and will have the command line that Swift
 tried to execute with.
+
+=== Dynamic profiles
+
+To set a profile setting based on the value of a Swift variable, you
+must use dynamic profiles.  This allows you to set profile settings in
+the +globus+ namespace.
+
+----
+app (file o) c(file i, int p)
+{
+  profile "mpi.processes" = 2+p;
+  profile "mpi.ppn" = 1;
+
+  my_mpi_program @i @o;
+}
+----
+
+This would be equivalent to the sites file settings:
+
+----
+<profile namespace="globus" key="mpi.processes">4</profile>
+<profile namespace="globus" key="mpi.ppn">1</profile>
+----
+
+except, of course, the number of MPI processes may not be dynamically
+set by the value of Swift variable +p+ in the sites file.
+
+Additional beneficial use cases of dynamic profiles may be to set the
++maxwalltime+ or +queue+ profile settings.
+
+=== Profile debugging
+
+Swift profiles, generally speaking, are converted into Java CoG
+"attributes", which are attached to each CoG task.  Thus, when reading
+the log or debugging, look for messages regarding "attributes".