[Swift-commit] r2398 - text/hpdc09submission
noreply at svn.ci.uchicago.edu
noreply at svn.ci.uchicago.edu
Tue Jan 6 11:52:09 CST 2009
Author: wilde
Date: 2009-01-06 11:52:07 -0600 (Tue, 06 Jan 2009)
New Revision: 2398
Modified:
text/hpdc09submission/paper.latex
Log:
Updated introduction. Adjusted author list (Added Mihael, Ian, Mike). Added comments for ACM bibliography style.
Modified: text/hpdc09submission/paper.latex
===================================================================
--- text/hpdc09submission/paper.latex 2009-01-05 20:40:02 UTC (rev 2397)
+++ text/hpdc09submission/paper.latex 2009-01-06 17:52:07 UTC (rev 2398)
@@ -3,25 +3,31 @@
\usepackage{graphicx}
\begin{document}
+\bibliographystyle{unsrt}
+% \bibliographystyle{abbrv} % for ACM SIGS style
+
\title{SwiftScript - a language for loosely coupled distributed
parallel scripting \\ draft - contact benc at ci.uchicago.edu}
% ACM styleguide says max 3 authors here, rest in acknowledgements
-\numberofauthors{3}
+\numberofauthors{4}
\author{
\alignauthor Ben Clifford \\
\affaddr{University of Chicago Computation Institute}\\
- \affaddr{5640 S. Ellis}\\
- \affaddr{Chicago, Illinois}\\
- \email{benc at ci.uchicago.edu} \alignauthor Other people \\
-\affaddr{elsewhere}
-\alignauthor Third author \\
-\affaddr{someplace}\\
-\affaddr{sometown}
+ \email{benc at ci.uchicago.edu}
+\alignauthor Ian Foster \\
+ \affaddr{University of Chicago Computation Institute}\\
+ \affaddr{Argonne National Laboratory}
+\alignauthor Mihael Hategan \\
+ \affaddr{University of Chicago Computation Institute}\\
+\and
+\alignauthor Michael Wilde \\
+ \affaddr{University of Chicago Computation Institute}\\
+ \affaddr{Argonne National Laboratory}
}
\maketitle
@@ -69,18 +75,27 @@
Swift's purpose is to enable ``loosely coupled scripting'' in a
convenient, powerful fashion. It is intended to serve as a higher
level framework for composing parallel pipelines of other programs and
-scripts - much like using a Makefile to encapsulate the compilation
+scripts. Much like a Makefile encapsulates the compilation
process, Swift puts a data-driven ``make-like'' wrapper around the
execution of programs (which can themselves be scripts written in any
other scripting language, or binary executables).
-Swift is data-oriented: the core of its programming model is to
-encapsulate the invocation of ``ordinary programs'' - technically, POSIX
-\emph{exec()} operations - in a manner that explicitly specifies the data
-objects (files) that are the inputs and outputs of each program
+As a ``parallel scripting
+language'', Swift is typically used to specify and execute scientific
+``workflows'' - which we define here as the execution of a
+series of steps to perform larger domain-specific tasks. We use the
+term workflow as defined by (Taylor et. al. 2006). So we often call a
+Swift script a workflow. FIXME: Drop this paragraph/concept? Or crisp it up.
+
+\subsection{Swift language concepts}
+
+The Swift programming model is data-oriented: it
+encapsulates the invocation of ``ordinary programs'' - technically, POSIX
+\emph{exec()} operations - in a manner that explicitly specifies the files
+and other arguments that are the inputs and outputs of each program
invocation. This formal but simple specification of data inputs and
outputs enables Swift to provide four critical features not provided
-by scripting languages like Perl, or Python, ``shells'', or Tcl:
+by scripting languages like Perl, Python, Tcl, or the various command-line ``shells'':
\begin{itemize}
\item It can provide location transparent execution: automatically
@@ -95,24 +110,15 @@
execution, so if interrupted or terminated, it can be restarted from
the point of interruption, without re-executing any work that was
logged as successfully completed.
-\item Swift variables can represent references to files, which are ten
-passed to application programs to operate on.
\end{itemize}
In the rest of this section, we provide an overview of Swift's main
-concepts. These concepts are elaborated, with examples, in subsequent
-sections. [Note: we will need to adjust between how much to specify
-here, and how mush to state just before each construct is introduced.
+concepts. Each concept is elaborated, with examples, in subsequent
+sections. [FIXME: we will need to adjust between how much to specify
+here, and how much to state just before each construct is introduced.
-\emph{The term ``workflow''.} Because Swift serves as a ``parallel scripting
-language'', it is typically used to specify and execute scientific
-``workflows'' - which we define here as the pattern of execution of a
-series of steps to perform larger domain-specific tasks. We use the
-term workflow as defined by (Taylor et. al. 2006). So we often call a
-Swift script a workflow.
-
-\emph{Dataset typing and mapping model}. Swift provides a high level
-representation of collections of data and how those collections are to
+\emph{Dataset typing and mapping model}. Swift provides for the high level
+specification of collections of data and of how those collections should
be processed by component programs. It provides a structured data-type
model for representing collections of files and directories that are
passed to Swift procedures, and a mapping model to convert between the
@@ -182,7 +188,11 @@
script are present in a single logical source file and are thus passed
to the Swift virtual machine all at once.
-\emph{Rationale}. Why do we need Swift? Why create yet another scripting
+\subsection{Rationale for creating Swift}
+
+\emph{FIXME: This section needs much polishing/condensing.}
+
+Why do we need Swift? Why create yet another scripting
language for the execution of application programs when so many exist?
Swift was developed to create a higher-level language that focuses not
on the details of executing sequences or ``pipelines'' of programs, but
@@ -191,15 +201,13 @@
the execution of, application pipelines that are not large-scale and
not necessarily distributed.
-\emph{FIXME: This section needs much polishing/condensing.}
-
Our motivation for developing Swift is based on the following premises:
-Scale (at least in the grid) requires distribution of execution among
+Scaling up requires the distribution of execution among
many computers (``resources''), and hence a ``grid'' approach. Even if a
single large parallel resource suffices, users won't always have
-access to the same one: resources are scarce, and users often need or
-want to utilize whatever resource happened to be free at the moment
+access to the same supercomputer cluster: resources are scarce, and users often need or
+want to utilize whatever resource happened to be available or economical at the moment
when they need to perform intensive computation.
While many application needs involve the execution of a single large
@@ -249,9 +257,9 @@
\section{The SwiftScript language}
-\subsection{Overview of the language}
+\subsection{Language basics}
- Programs written in the SwiftScript language are called (Swift)
+Programs written in the SwiftScript language are called (Swift)
scripts. A Swift script describes the interface of each component
program, and how those components communicate with each other.
Component programs in a Swift script are coupled together by
@@ -1109,9 +1117,11 @@
defining; specifically for CoG, need to declare that it builds on top
of that; relation to old old VDL2 papers (eg that Yong was on...)
- some dude did some stuff about BOINC - that could have a one-liner
+ some dude (it was Xu Du) did some stuff about BOINC - that could have a one-liner
if it was actually written up somewhere; otherwise ignore.
+Not likely that it was written up but I will ask. (mike)
+
performance: application tuning graphs; provisioning and coaster
file access (give one-liner numbers for those); file system layout
tuning to accomodate GPFS - can make before/after one-liners for that
@@ -1124,10 +1134,13 @@
Swift core: me, wilde, hategan, milena, yong, ian
CNARI: skenny
OSG: mats
-Site selection: xi li, dude-who-did-data-affinity
+Site selection: xi li, ragib
+App installed: Zhengxiong Howe
Falkon: Ioan, zhao
-Collective IO: allan
+Collective IO: allan, zhao, ioan
+Users: Uri, Kubal, Hocky, UMD Student, ...
+
more explicit mapper description should include table of all/common mappers
ramble about separation of parallel execution concerns and dataflow spec
@@ -1167,5 +1180,6 @@
\bibitem{GPFS} - GPFS
\end{thebibliography}
+
+% \bibliography{paper} % for ACM SIGS style
\end{document}
-
More information about the Swift-commit
mailing list