<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
You say that you use UDP on the workers. This might be more light
weight, but might also pose practical issues. <br>
<br>
Some of those are:<br>
- might not work well on any network other than a LAN<br>
- won't be friendly to firewalls or NATs, no matter if you the service
pushes jobs, or workers pull jobs; the logic is that you need 2 way
communication, and using UDP (being a connectionless protocol), its
like having a server socket and a client socket on both ends of the
communication at the same time. This might not matter if the service
and the worker are on the same LAN with no NATs or firewalls in the
middle, but, it would matter on a machine such as the BG/P, as there is
a NAT inbetween the login nodes and the compute nodes. In essence, for
this to work on the BG/P, you'll need to avoid having server side
sockets on the compute nodes (workers), and you'll probably only be
able to do that via a connection oriented protocol (i.e. TCP). Is
switching to TCP a relatively straight forward option? If not, it
might be worth implementing to make the implementation more flexible<br>
- loosing messages and recovering from them will likely be harder than
anticipated; I have a UDP version of the notification engine that
Falkon uses, and after much debugging, I gave up and switched over to
TCP. It worked most of the time, but the occasional lost message (1 in
1000s, maybe even more rare) made Falkon unreliable, and hence I
stopped using it. <br>
<br>
Is the 180 tasks/sec the overall throughput measured from Swift's point
of view, including overhead of wrapper.sh? Or is that a
micro-benchmark measuring just the coaster performance? <br>
<br>
Ioan<br>
<br>
<br>
Mihael Hategan wrote:
<blockquote cite="mid:1207311167.7171.12.camel@blabla.mcs.anl.gov"
type="cite">
<pre wrap="">On Fri, 2008-04-04 at 06:59 -0500, Michael Wilde wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Mihael, this is great progress - very exciting.
Some questions (dont need answers right away):
How would the end user use it? Manually start a service?
Is the service a separate process, or in the swift jvm?
</pre>
</blockquote>
<pre wrap=""><!---->
I though the lines below answered some of these.
A user would specify the coaster provider in sites.xml. The provider
will then automatically deploy a service on the target machine without
the user having to do so. Given that the service is on a different
machine than the client, they can't be in the same JVM.
</pre>
<blockquote type="cite">
<pre wrap="">How are the number of workers set or adjusted?
</pre>
</blockquote>
<pre wrap=""><!---->
Currently workers are requested as much as needed, up to a maximum. This
is preliminary hence "Better allocation strategy for workers".
</pre>
<blockquote type="cite">
<pre wrap="">Does a service manage workers on one cluster or many?
</pre>
</blockquote>
<pre wrap=""><!---->
One service per cluster.
</pre>
<blockquote type="cite">
<pre wrap="">At 180 jobs/sec with 10 workers, what were the CPU loads on swift,
worker and service?
</pre>
</blockquote>
<pre wrap=""><!---->
I faintly recall them being at less than 50% for some reason I don't
understand.
</pre>
<blockquote type="cite">
<pre wrap="">Do you want to try this on the workflows we're running on Falkon on the
BGP and SiCortex?
</pre>
</blockquote>
<pre wrap=""><!---->
Let me repeat "prototype" and "more testing". In no way do I want to do
preliminary testing with an application that is shaky on an architecture
that is also shaky.
Mihael
</pre>
<blockquote type="cite">
<pre wrap="">Im eager to try it when you feel its ready for others to test.
Nice work!
- Mike
On 4/4/08 4:39 AM, Mihael Hategan wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I've been asked for a summary of the status of the coaster prototype, so
here it is:
- It's a prototype so bugs are plenty
- It's self deployed (you don't need to start a service on the target
cluster)
- You can also use it while starting a service on the target cluster
- There is a worker written in Perl
- It uses encryption between client and coaster service
- It uses UDP between the service and the workers (this may prove to be
better or worse choice than TCP)
- A preliminary test done locally shows an amortized throughput of
around 180 jobs/s (/bin/date). This was done with encryption and with 10
workers. Pretty picture attached (total time vs. # of jobs)
To do:
- The scheduling algorithm in the service needs a bit more work
- When worker messages are lost, some jobs may get lost (i.e. needs more
fault tolerance)
- Start testing it on actual clusters
- Do some memory consumption benchmarks
- Better allocation strategy for workers
Mihael
------------------------------------------------------------------------
_______________________________________________
Swift-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a>
<a class="moz-txt-link-freetext" href="http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel">http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel</a>
</pre>
</blockquote>
</blockquote>
<pre wrap=""><!---->
_______________________________________________
Swift-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a>
<a class="moz-txt-link-freetext" href="http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel">http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: <a class="moz-txt-link-abbreviated" href="mailto:iraicu@cs.uchicago.edu">iraicu@cs.uchicago.edu</a>
Web: <a class="moz-txt-link-freetext" href="http://www.cs.uchicago.edu/~iraicu">http://www.cs.uchicago.edu/~iraicu</a>
<a class="moz-txt-link-freetext" href="http://dev.globus.org/wiki/Incubator/Falkon">http://dev.globus.org/wiki/Incubator/Falkon</a>
<a class="moz-txt-link-freetext" href="http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page">http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page</a>
===================================================
===================================================
</pre>
</body>
</html>