<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
<br>
<br>
Ben Clifford wrote:
<blockquote
cite="mid:Pine.LNX.4.64.0803070831480.32747@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">
On Fri, 7 Mar 2008, Ioan Raicu wrote:
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">I notice you're using quite an old version of swift
</pre>
</blockquote>
<pre wrap=""><a class="moz-txt-link-abbreviated" href="mailto:iraicu@login1.surveyor:/home/zzhang/cog/modules/vdsk">iraicu@login1.surveyor:/home/zzhang/cog/modules/vdsk</a>> svn info
</pre>
</blockquote>
<pre wrap=""><!---->
</pre>
<blockquote type="cite">
<pre wrap="">Revision: 1673
</pre>
</blockquote>
<pre wrap=""><!---->
Zhao's log reported this:
<a class="moz-txt-link-abbreviated" href="mailto:zhaozhang@viper:~/vdsk-0.3/examples/vdsk">zhaozhang@viper:~/vdsk-0.3/examples/vdsk</a>> swift first.swift
Swift v0.3 r1319 (modified locally)
</pre>
</blockquote>
Notice that this Swift is from viper, a machine in RI. He might be
playing with this to get used to learning about Swift, but that is not
being used on the BG/P in any way, as everything we run on the BG/P
needs to run from the login nodes. Zhao, can you confirm that the runs
you are making with Swift on the BG/P are indeed using a recent
version? Can you also do an update of Swift on the BG/P to make sure
we have the latest?<br>
<blockquote
cite="mid:Pine.LNX.4.64.0803070831480.32747@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">
which is nowhere near the version that you report - something in the 1600s
is a reasonable number, but that's not what the log output was. So please
clarify which version you actually have poor behaviour for. If it is in
the r1600 range, then I'm interested to fix up - but in the r1600s there
should be absolutely no cross-node shared log files at all.
</pre>
<blockquote type="cite">
<pre wrap="">Every job still has a scratch space sandbox, which results in a mkdir,
symbolic linking, and finally a cleanup remove dir. I think this is the
dir he is referring to. BTW, if there would be an easy way to eliminate
this entire mkdir part of the wrapper script without breaking anything
in Swift, it would be nice. The apps we are dealing with don't need the
sandboxing, as we know all input files, and all output files, and we'll
never have input as *.fits that might be ambiguous if we don't sandbox.
</pre>
</blockquote>
<pre wrap=""><!---->
If the jobs touch only those files, I think you can probably eliminate the
mkdir, the ln to copy files in and the cp to copy files back out, and run
the job in the shared directory directly. However, as each node will then
be trying to do stuff to that shared directly, my initial thoughts would
be that it wouldn't really change much (perhaps better, perhaps worse).
</pre>
</blockquote>
That is what I thought. We'll try that and see what we get! We'll
keep you posted.<br>
<br>
Ioan<br>
<br>
<pre class="moz-signature" cols="72">--
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: <a class="moz-txt-link-abbreviated" href="mailto:iraicu@cs.uchicago.edu">iraicu@cs.uchicago.edu</a>
Web: <a class="moz-txt-link-freetext" href="http://www.cs.uchicago.edu/~iraicu">http://www.cs.uchicago.edu/~iraicu</a>
<a class="moz-txt-link-freetext" href="http://dev.globus.org/wiki/Incubator/Falkon">http://dev.globus.org/wiki/Incubator/Falkon</a>
<a class="moz-txt-link-freetext" href="http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page">http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page</a>
===================================================
===================================================
</pre>
</body>
</html>