[Swift-devel] Swift-issues (PBS+NFS Cluster)

[Ben Clifford]

On Mon, 11 May 2009, yizhu wrote:

> I think my next step is to write a sample swift code to check if we can let
> swift grab input files from S3, execute it, and then write the output files
> back to S3.

Swift won't be able to do this out of the box.

In the present model, you must have a shared filesystem which is 
accessible from the Swift client and from each worker.

I've looked at making the Swift worker-side script, libexec_swiftwrap able 
to invoke arbitrary commandlines to pull in data instead of using posix cp 
or ln. You would need commandline tools to transfer a file from S3 to the 
local system, and from the local system to S3. Do you have those commands? 
What do they look like?

> Since each Amazon virtual node only has limited storage space( Small instance
> for 1.7GB, Large Instance for 7.5GB), we may need to use EBS(Elastic Block
> Store) to storage temp files created by swift. The EBS behaved like a hard
> disk volume and can be mounted by any virtual nodes. But here arise a problem,
> since a volume can only be attached to one instance at a time[1], so the files
> store in EBS mounted by one node won't be shared by any other nodes; we lost
> the file sharing ability which I think is a fundamental requirement for swift.

For getting a probably poorly performing prototype, the fuse idea 
mentioned elsewhere seems interesting. From my understanidng, you should 
be able to mount an EBS volume on the cluster head node and re-export it 
using the existing NFS shared file system.

--