<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Sorry, Ben.<br>
<br>
I didn't save the swift log file. If you really need the old -info
file, I could redo the test, and try to send them to you.<br>
But for now, I have several urgent issues.<br>
<br>
zhao<br>
<br>
Ben Clifford wrote:
<blockquote
cite="mid:Pine.LNX.4.64.0804031005560.9854@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">I just asked zhao for the log files (both swift and -info) for the patched
run; but I think I'd like to see the unpatched run logs too.
On Wed, 2 Apr 2008, Ioan Raicu wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Ben,
Thanks again for the patches, they made a huge difference, increased
efficiency from 21% to 81%!
Here are the numbers:
1 Node Perf Falkon Swift+Falkon Swift+Falkon (patched)
Min 63.618 53.782 169.139 58.538
Average 64.76 65.47253 309.1945 80.21246
Median 64.74072 64.774 313.5535 76.5245
Max 65.863 94.447 605.654 115.237
Standard Deviation 0.488984 3.863944 52.13821
10.95652
Efficiency 100% 99% 21% 81%
The first column shows the per task statistic when running on 1 node (4 CPUs)
through Falkon. The second column are the statistics for running the
application at large scale, on 2048 CPUs. The 3rd column is running
Swift+Falkon (both from SVN) on 256 CPUs. The 4th column is Swift+Falkon, but
Swift has the 3 patches applied. Essentially, the per task execution time was
reduced from 309 seconds to 80 seconds, where the ideal would have been 64
seconds. It brought the efficiency from 21% to 81% for this particular
workload. This looks fantastic!
We'll have to verify that we can maintain this 81% efficiency to higher number
of CPUs. In the meantime, if you can think of anything else that we could do
to keep pushing the 81% efficiency number higher, let us know.4
Thanks again,
Ioan
Ben Clifford wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Mon, 31 Mar 2008, Ben Clifford wrote:
</pre>
<blockquote type="cite">
<pre wrap="">This temporary directory handling is pretty ugly - it should be a couple
lines change to wrapper.sh to get similar functionality using the existing
swift temporary direcotry handling - change the path to /tmp and use cp
instead of ln -s. That way you can take advantage of Swift's existing
unique job IDs and error handling too.
</pre>
</blockquote>
<pre wrap="">Attached are three patches that will apply against svn r1775:
The first puts temporary directories in /tmp rather than on shared fs.
<a class="moz-txt-link-freetext" href="http://www.ci.uchicago.edu/~benc/tmp/wrapper-tmp-dirs-on-tmp">http://www.ci.uchicago.edu/~benc/tmp/wrapper-tmp-dirs-on-tmp</a>
The second copies the application file to the worker in each job execution
(though doesn't do any worker-node caching of such between jobs)
<a class="moz-txt-link-freetext" href="http://www.ci.uchicago.edu/~benc/tmp/wrapper-tmp-dirs-mv-executable">http://www.ci.uchicago.edu/~benc/tmp/wrapper-tmp-dirs-mv-executable</a>
The third creates the worker node log on /tmp and copies it at the end.
<a class="moz-txt-link-freetext" href="http://www.ci.uchicago.edu/~benc/tmp/wrapper-tmp-log-locally">http://www.ci.uchicago.edu/~benc/tmp/wrapper-tmp-log-locally</a>
The three modify all wrapper.sh and should be applied in the above order.
With the first two patches, the timestamps in the usual info logs will
provide information about how long the copies take, in the same way that
they usually indicate times for other execution stages.
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap=""><!---->
</pre>
</blockquote>
</body>
</html>