[Swift-devel] GPFS issue of SWIFT on BGP

[Ben Clifford]

On Tue, 10 Feb 2009, Zhao Zhang wrote:

> Now, we start swift on Login Node, and the working directory will be created
> on IO Node, so that all intermediate files and final
> result files will be copied back to Login Node(GPFS) once they are generated.
> Here we got an old problem, all IO nodes are trying
> to write files in the same directory, which we are trying to avoid all the
> way.
> My solution would be modify the ssh-provider source code, implement an
> asynchronous collector logic there.

Can you describe what is going on here more explicitly.

How do multiple IO nodes end up writing to the same GPFS directory?

It is unclear to me from what you write how that comes about - as I 
understand it:

 . submit side data files are posix-accessed only by the swift submit-side 
client

 . files on the I/O nodes (the remote sites) use pset-local storage

 . any communication between the I/O nodes and submit-side client happens 
over ssh.

Where does an I/O node access machine-wide GPFS?

--