<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
<br>
<br>
Ben Clifford wrote:
<blockquote
cite="mid:Pine.LNX.4.64.0812012225230.4020@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">On Mon, 1 Dec 2008, Zhao Zhang wrote:
</pre>
...<br>
<blockquote type="cite">
<pre wrap="">3. Collective IO
Build a shared file system which could be accessed by all CN, instead of
writing output data to GPFS, workers
copy intermediate output data to this shared ram-disk. And retrieve the data
from IFS.
Several Concerns:
a) reliability of torus network --- we need to test more about this.
b) performance of torus network --- could this be really performing better
than GPFS? If not, at what scale
could torus perform better than GPFS?
</pre>
</blockquote>
<pre wrap=""><!---->
As phrased above, this seems a little strange:
"rather than use a shared file system, lets build a shared file system and
use it."
</pre>
</blockquote>
Zhao meant that instead of using 1 large global shared file system,
lets build many smaller shared file systems, that can keep traffic
localized within the machine (i.e. per pset) to allow relatively linear
scalability with every additional processing power and network
bandwidth. Depending at what level we are working at, we might also be
able to relax some of the semantics of the POSIX file system, in order
to improve performance and scalability, at the expense of other things.<br>
<br>
Ioan<br>
<blockquote
cite="mid:Pine.LNX.4.64.0812012225230.4020@dildano.hawaga.org.uk"
type="cite">
<pre wrap="">
Do you mean building some general purpose posix shared file system? If so,
this seems quite hard, and seems directly in competition with PVFS and
GPFS, a competition which you are pretty much guaranteed to lose.
It may be that you mean something completely different - your concerns
about the torus network seem unrelated to writing a posix fs, so I think
that may be the case (or maybe you are overspecialising your concerns).
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: <a class="moz-txt-link-abbreviated" href="mailto:iraicu@cs.uchicago.edu">iraicu@cs.uchicago.edu</a>
Web: <a class="moz-txt-link-freetext" href="http://www.cs.uchicago.edu/~iraicu">http://www.cs.uchicago.edu/~iraicu</a>
<a class="moz-txt-link-freetext" href="http://dev.globus.org/wiki/Incubator/Falkon">http://dev.globus.org/wiki/Incubator/Falkon</a>
<a class="moz-txt-link-freetext" href="http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page">http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page</a>
===================================================
===================================================
</pre>
</body>
</html>