[Swift-devel] several alternatives to design the data management system for Swift on SuperComputers

[Ioan Raicu]

But its not just about directories and GPFS locking.... its about 8 or 
16 large servers with 10Gb/s network connectivity (as is the case for 
GPFS) compared to potentially 40K servers, each with 1Gb/s connectivity 
(as would be the case in our example).  The potential raw throughput of 
the later case, when we use all 40K nodes as servers to the file system, 
is orders of magnitude larger than a static configuration with 8 or 16 
servers.  Its not yet clear we can actually achieve anything close to 
the upper bound of performance at full scale, but it should be obvious 
that the performance characteristics will be quite different between 
GPFS and CIO.

Ioan

Mihael Hategan wrote:
> On Mon, 2008-12-01 at 17:10 -0600, Ioan Raicu wrote:
>   
>> Mihael Hategan wrote: 
>>     
>>> On Mon, 2008-12-01 at 16:52 -0600, Ioan Raicu wrote:
>>>
>>>   
>>> ...
>>>
>>>   
>>>       
>>>> I don't think you realize how expensive GPFS access is when doing so
>>>> at 100K CPU scale.
>>>>     
>>>>         
>>> I don't think I understand what you mean by "access". As I said, things
>>> that generate contention are going to be slow.
>>>
>>> If the problem requires that contention to happen, then it doesn't
>>> matter what the solution is. If it does not, then I suspect that there
>>> is a way to avoid contention in GPFS, too (sticking things in different
>>> directories).
>>>   
>>>       
>> The basic idea is that many smaller shared file systems will scale
>> better than 1 large file system, as the contention is localized.
>>     
>
> Which is the same behaviour you get if you have a hierarchy of
> directories. This is what Ben implemented in Swift.
>
>   
>>  The problem is that having 1 global namespace is simple and straight
>> forward, but having N local namespaces is not, and requires extra
>> management.
>>     
>
> Right. That's why most filesystems I know of treat directories as
> independent files containing file metadata (aka. "local namespaces").
>
>
>   

-- 
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
http://dev.globus.org/wiki/Incubator/Falkon
http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
===================================================
===================================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20081201/c8df0399/attachment.html>