<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
<br>
<br>
Ben Clifford wrote:
<blockquote
cite="mid:Pine.LNX.4.64.0803062323120.32747@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">you should send questions like this to swift-devel or swift-user list
rather than attempting to compose your own list of likely candidates and
witholding the information from the public archives.
</pre>
</blockquote>
Made the reply to the Swift devel mailing lists...<br>
<blockquote
cite="mid:Pine.LNX.4.64.0803062323120.32747@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">I am trying to dig into the wrapper.sh, disable the log to enhance the
performance.
</pre>
</blockquote>
<pre wrap=""><!---->
Do you have numbers that suggest logging is causing a performance
degradation? </pre>
</blockquote>
By default, Swift is able to do about 5 jobs/sec running over Falkon on
256 CPUs on the BG/P, where each job is a sleep 0. The Falkon command
line client can do about 1700 jobs/sec on the same hardware. 9 months
ago, I saw Swift go from a few jobs/sec to about 50 jobs/sec by
stripping out all logging (i.e. echo "..." >> LOG) from the
wrapper script, and by removing the mkdir and symbolic linking. Since
the mkdir is much improved now, I assume that is not the bottleneck,
but doing 10~20 echo to a log file on the shared file system from many
nodes at the same time is expensive, which I think is the main
bottleneck in the current wrapper script. Once Zhao is done disabling
all logging, except for necessary ones, we'll have a better idea of how
fast we can go, and if it is necessary to eliminate the mkdir step as
well. I think getting about 50 jobs/sec is within reach by
streamlining the wrapper.sh script, but I think we'll have to think of
ways to push those numbers even higher!<br>
<blockquote
cite="mid:Pine.LNX.4.64.0803062323120.32747@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">I notice you're using quite an old version of swift </pre>
</blockquote>
<a class="moz-txt-link-abbreviated" href="mailto:iraicu@login1.surveyor:/home/zzhang/cog/modules/vdsk">iraicu@login1.surveyor:/home/zzhang/cog/modules/vdsk</a>> svn info<br>
Path: .<br>
URL: <a class="moz-txt-link-freetext" href="https://svn.ci.uchicago.edu/svn/vdl2/trunk">https://svn.ci.uchicago.edu/svn/vdl2/trunk</a><br>
Repository Root: <a class="moz-txt-link-freetext" href="https://svn.ci.uchicago.edu/svn/vdl2">https://svn.ci.uchicago.edu/svn/vdl2</a><br>
Repository UUID: e2bb083e-7f23-0410-b3a8-8253ac9ef6d8<br>
Revision: 1673<br>
Node Kind: directory<br>
Schedule: normal<br>
Last Changed Author: <a class="moz-txt-link-abbreviated" href="mailto:benc@CI.UCHICAGO.EDU">benc@CI.UCHICAGO.EDU</a><br>
Last Changed Rev: 1670<br>
Last Changed Date: 2008-02-09 12:42:56 -0600 (Sat, 09 Feb 2008)<br>
<br>
It doesn't seem that old, but we'll update to the latest one before we
do more experiments.<br>
<blockquote
cite="mid:Pine.LNX.4.64.0803062323120.32747@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">(the last
release) - we made substantial log speed improvements subsequent to that.
If you're hitting log file problems here, there is a fair chance that
you'll encounter other scalability problems on the site filesystem that we
also fixed in SVN some months ago.
</pre>
</blockquote>
Right, I know, and I thought we were using a late enough version that
had those fixes. Just to be sure, we'll upgrade!<br>
<blockquote
cite="mid:Pine.LNX.4.64.0803062323120.32747@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">One thing I notice is that for each job, correct me if I am wrong, SWIFT
will make a unique directory with the date and a random string, then
copy wrapper.sh and other necessary files to that directory.
</pre>
</blockquote>
<pre wrap=""><!---->
It should do that one per workflow per site, not per job.
</pre>
</blockquote>
Every job still has a scratch space sandbox, which results in a mkdir,
symbolic linking, and finally a cleanup remove dir. I think this is
the dir he is referring to. BTW, if there would be an easy way to
eliminate this entire mkdir part of the wrapper script without breaking
anything in Swift, it would be nice. The apps we are dealing with
don't need the sandboxing, as we know all input files, and all output
files, and we'll never have input as *.fits that might be ambiguous if
we don't sandbox.<br>
<br>
Ioan<br>
<blockquote
cite="mid:Pine.LNX.4.64.0803062323120.32747@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap=""> echo -abc "Hello, world!" stdout=@filename(t);
</pre>
</blockquote>
<pre wrap=""><!---->
Put the -abc in quotes:
echo "-abc" "hello"
to solve the immediate problem.
However, note that the command:
echo -abc hello
executes successfully on my linux and os x boxes.
If you want a job that will fail, try the 'false' command.
</pre>
<blockquote type="cite">
<pre wrap="">RunID: 20080306-1647-4nd1cymf
Execution failed:
Variable not found: abc
</pre>
</blockquote>
<pre wrap=""><!---->
This is because you did not quote "-abc", so swift is trying to give you
the unary negative value of -abc (just like if you said -abc in Java or
C).
</pre>
<blockquote type="cite">
<pre wrap="">But I still can not find the default working directory of this task.
Also, I know there is a log file for this wrapper, so it is in the
working directory, right?
</pre>
</blockquote>
<pre wrap=""><!---->
Swift will never have attempted to run the above, because of the above
error.
</pre>
<blockquote type="cite">
<pre wrap="">Another question is, could you give me a simple task description of
wrapper.sh? So I could invoke wrapper.sh directly without falkon. I got
a task description before,
140.221.82.10 : urn:0-195-1203621652641 : EXECUTABLE /bin/bash ARGUEMENTS
shared/wrapper.sh sleep-1j38kqoi -jobdir 1 -e /bin/sleep -out stdout.txt -err
stderr.txt -i -d -if -of -k -a 0
but it is within the working directory, and I don't understand what
"sleep-1j38kqoi" means.
</pre>
</blockquote>
<pre wrap=""><!---->
sleep-1j38kqoi is a job identifier (in Swift internal language, an
execute2 identifier, perhaps) which identifies one attempt to run an
application. This is used to label log files and working directories for
this.
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: <a class="moz-txt-link-abbreviated" href="mailto:iraicu@cs.uchicago.edu">iraicu@cs.uchicago.edu</a>
Web: <a class="moz-txt-link-freetext" href="http://www.cs.uchicago.edu/~iraicu">http://www.cs.uchicago.edu/~iraicu</a>
<a class="moz-txt-link-freetext" href="http://dev.globus.org/wiki/Incubator/Falkon">http://dev.globus.org/wiki/Incubator/Falkon</a>
<a class="moz-txt-link-freetext" href="http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page">http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page</a>
===================================================
===================================================
</pre>
</body>
</html>