<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Where can I get the Swift logs for these runs, I'd like to pass them
through my Swift-to-Falkon conversion tool :)<br>
<br>
Ioan<br>
<br>
Mihael Hategan wrote:
<blockquote cite="mid:1256621747.17056.67.camel@localhost" type="cite">
<pre wrap="">On Mon, 2009-10-26 at 23:19 -0500, Ioan Raicu wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
Mihael Hategan wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Mon, 2009-10-26 at 16:36 -0500, Ioan Raicu wrote:
[...]
</pre>
</blockquote>
</blockquote>
<pre wrap=""><!---->[...]
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">OK. So which plot/data should I be looking at to get the summary of
the per tasks performance?
</pre>
</blockquote>
<pre wrap=""><!---->
None. You calculate the efficiency based on the total time, individual
task time and number of nodes.
</pre>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">64k jobs, 4000 workers:
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/~hategan/report-dc-4000/">http://www.mcs.anl.gov/~hategan/report-dc-4000/</a>
</pre>
</blockquote>
<pre wrap="">Shortest event (s): 106.119999885559
Longest event (s): 1246.60699987411
Mean event duration (s): 334.987874176266
Standard deviation of event duration (s): 290.212811366649
</pre>
</blockquote>
</blockquote>
</blockquote>
<pre wrap=""><!---->[...]
</pre>
<blockquote type="cite">
<pre wrap="">In the case of the above 18%, I took 60 / 334 ~ 0.18 = 18%
</pre>
</blockquote>
<pre wrap=""><!---->
See <a class="moz-txt-link-freetext" href="http://en.wikipedia.org/wiki/Speedup">http://en.wikipedia.org/wiki/Speedup</a>
Efficiency is speedup divided by number of cores. You cannot infer
things from the mean duration because it says nothing about the degree
of parallelism. I.e. you're not interested in how fast individual things
are going, but how fast all things are going overall. That's why you can
have a crappy-cpu machine like the BGP be the in the top 10. Allow me to
paint this:
Scenario 1: [-1-][-2-][-1-][-2-]
Scenario 2: [---1----][---2----]
(how two tasks are scheduled by a scheduler on one CPU).
Efficiency is the same in both cases. In scenario 1 average task
duration is 15 characters, in scenario 2 it's 10 characters. In both
cases the raw task duration is 10 characters.
[...]
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">I see the block utilization near 100% all the time,
</pre>
</blockquote>
<pre wrap=""><!---->
Right. That's because it measures the wrapper time, not the wrapped job
time (some time is spent doing whatever the wrapper does). That just
says that the job-to-worker dispatch algorithm in the coasters works ok
with that load.
</pre>
<blockquote type="cite">
<pre wrap=""> so that doesn't seem to match the other data I saw.
</pre>
</blockquote>
<pre wrap=""><!---->
They measure different things. But if you calculate the efficiency the
proper way, you'll see that they are closer.
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">2. Multiply 60s with the number of jobs (65535), divide by the number of
workers (6*1024) and then by the total time since the first job starts
to when the last job finishes (or you could choose the middle of the
ramp-up to the middle of the ramp-down to get some sort of amortized
efficiency). That gives you about 91% end-to end and 96% amortized. Or
you could divide by the total time, including swift startup, partition
boot time, etc. to get 64%.
</pre>
</blockquote>
<pre wrap="">65535*60/(6*1024) ~ 640 sec. I see the end-to-end time being about
1300 sec, or 1100 sec if we look at just Karajan. The 64% efficiency
is in the ballpark, but I don't see where the 91% and 96% are coming
from.
</pre>
</blockquote>
<pre wrap=""><!---->
I think you're mixing the runs. Sorry I didn't make it more clear, but
dc-4000 is the 4*1024 core run and dc-6000 is the 6*1024 core run. So
you're dividing the 4*1024 core speedup by 6*1024 which gives you 2/3 of
the efficiency. Multiply 64% by 3/2 to get the proper number back.
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
=================================================================
Ioan Raicu, Ph.D.
NSF/CRA Computing Innovation Fellow
=================================================================
Center for Ultra-scale Computing and Information Security (CUCIS)
Department of Electrical Engineering and Computer Science
Northwestern University
2145 Sheridan Rd, Tech M384
Evanston, IL 60208-3118
=================================================================
Cel: 1-847-722-0876
Tel: 1-847-491-8163
Email: <a class="moz-txt-link-abbreviated" href="mailto:iraicu@eecs.northwestern.edu">iraicu@eecs.northwestern.edu</a>
Web: <a class="moz-txt-link-freetext" href="http://www.eecs.northwestern.edu/~iraicu/">http://www.eecs.northwestern.edu/~iraicu/</a>
<a class="moz-txt-link-freetext" href="https://wiki.cucis.eecs.northwestern.edu/">https://wiki.cucis.eecs.northwestern.edu/</a>
=================================================================
=================================================================
</pre>
</body>
</html>