<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi,<br>
There are several papers out there from our group that shows different
aspects of performance of Swift. Here are a few:<br>
<ul>
<li><a class="moz-txt-link-freetext"
href="http://people.cs.uchicago.edu/%7Eiraicu/publications/2008_NOVA08_book-chapter_Swift.pdf">http://people.cs.uchicago.edu/~iraicu/publications/2008_NOVA08_book-chapter_Swift.pdf</a></li>
<ul>
<li>Figure 9: Shows the memory footprint per job (aka tasks, or
nodes in the DAG graph)</li>
<ul>
<li>shows memory footprint of 3.2KB per node</li>
</ul>
<li>Figure 18: Shows a large scale application</li>
<ul>
<li>20K tasks on 200 CPUs with average task lengths of 200
seconds is a comfortable range for Swift and Falkon</li>
<li>we have more recent results, not published yet, that has 16K
tasks on 2048 CPUs with an average task length of 87 seconds which
worked well<br>
</li>
</ul>
</ul>
<li><a class="moz-txt-link-freetext"
href="http://people.cs.uchicago.edu/%7Eiraicu/publications/2007_SWF07_Swift.pdf">http://people.cs.uchicago.edu/~iraicu/publications/2007_SWF07_Swift.pdf</a></li>
<ul>
<li>Figure 6: Shows the speedup achieved with different task lengths</li>
<ul>
<li>Conclusion is that using multi-level scheduling with the
Falkon provider, even tasks in the range of seconds long can achieve
good speedup<br>
</li>
</ul>
<li>Figure 7: Shows the throughput in tasks/sec achieved by Swift</li>
<ul>
<li>shows Swift achieving 50+ tasks/sec throughputs using Falkon<br>
</li>
<li>the paragraph right after this figure mentions that Swift
running directly with GRAM2 and PBS can achieve 2 jobs/sec; the
implication of this is that jobs typically take 15~60 seconds to
startup, which reflects the cost of scheduling, scheduling cycles, and
local resource manager's (LRM) time to setup the remote nodes; there
are also limitations on how many jobs can be submitted at a time, as
each job queued might consume some resources on the LRM, or there might
be policies in place that limit the number of jobs that can be queued;
this means that aggressive throttling must take place, which in
practice, reduces the sustained rate that Swift can submit/execute jobs
to a single site, to even lower than 2 jobs/sec</li>
</ul>
</ul>
</ul>
So, to answer you question, the performance of Swift (and any other
workflow system) will heavily rely on how efficient you can dispatch
jobs/tasks to remote resources, how long jobs/tasks are, how data
intensive the application is, and how much data movement must happen
before the job runs and after. If you have a fast enough file system,
and your application execution times are small, you can expect anywhere
from 1 to 50 jobs/sec from Swift, depending on what technologies you
use to interface between Swift and the remote resources (e.g. GRAM,
PBS, Condor, Falkon, etc).<br>
<br>
Cheers,<br>
Ioan <br>
<br>
<br>
J A wrote:
<blockquote
cite="mid:b19f113c0806021620v48c5a116r6f990f43877b899b@mail.gmail.com"
type="cite">
<div>Hi All:</div>
<div> </div>
<div>
<p style="margin: 0in 0in 0pt;"><font face=""><font size="2">Based on
my reading, the performance from execution a swift workflow depends on
the parallelism<span> </span>that a workflow has.</font></font></p>
<p style="margin: 0in 0in 0pt;"><font face="" size="2"> </font></p>
<p style="margin: 0in 0in 0pt;"><font face=""><font size="2">If I
have a workflow that contains several processors where each processor
(procedure) depends on the previous one (output of a processor "A" is
the input for processors "B" and so on.)</font></font></p>
<p style="margin: 0in 0in 0pt;"><font face="" size="2"> </font></p>
<p style="margin: 0in 0in 0pt;"><font face=""><font size="2">How the
performance of using swift in this case compare to other systems that
execute workflows where there isn't any parallelism in the workflow?</font></font></p>
<p style="margin: 0in 0in 0pt;"><font face="" size="2"> </font></p>
<p style="margin: 0in 0in 0pt;"><font face="" size="2">-- <br>
Thanks,<br>
<span></span><span class="sg"><br>
Jamal </span></font></p>
</div>
<pre wrap=""><hr size="4" width="90%">
_______________________________________________
Swift-user mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a>
<a class="moz-txt-link-freetext"
href="http://mail.ci.uchicago.edu/mailman/listinfo/swift-user">http://mail.ci.uchicago.edu/mailman/listinfo/swift-user</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: <a class="moz-txt-link-abbreviated"
href="mailto:iraicu@cs.uchicago.edu">iraicu@cs.uchicago.edu</a>
Web: <a class="moz-txt-link-freetext"
href="http://www.cs.uchicago.edu/%7Eiraicu">http://www.cs.uchicago.edu/~iraicu</a>
<a class="moz-txt-link-freetext"
href="http://dev.globus.org/wiki/Incubator/Falkon">http://dev.globus.org/wiki/Incubator/Falkon</a>
<a class="moz-txt-link-freetext"
href="http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page">http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page</a>
===================================================
===================================================
</pre>
</body>
</html>