[Swift-devel] Swift-issues (PBS+NFS Cluster)

[Michael Wilde]

That all sounds very good.

On 5/7/09 12:17 PM, Ben Clifford wrote:
> On Thu, 7 May 2009, Michael Wilde wrote:
> 
>> I wonder if the Swift engine could pull the data back from a GridFTP server
>> running on the worker node?
> 
> There's nothing virtual-machine specific here, I think.
> 
> From the perspective of the cio work:
> 
> If the site shared filesystem is some distributed fs (which is what I 
> think the collective I/O stuff mostly is) then a CIO-aware gridftp client 
> could choose one of several gridftp servers that have access to the same 
> cio file system, and select a preferable one to contact.
> 
> That could fit into Swift at the cog file transfer layer, as an 
> alternative provider to gridftp.
> 
> It would preserve Swift's view of sites as fairly monolithic entities, 
> with intra-site selection activity made by lower levels in the stack (as 
> happens with pbs or falkon or coasters).
> 
> That hierarchy feels more scalable to me - it works ok in the job 
> submission scenario with worker nodes being allocated by LRMs, for 
> example.
>