[Swift-devel] GPFS issue of SWIFT on BGP

[Zhao Zhang]

Hi,

Ben Clifford wrote:
> On Tue, 10 Feb 2009, Zhao Zhang wrote:
>
>   
>> Now, we start swift on Login Node, and the working directory will be created
>> on IO Node, so that all intermediate files and final
>> result files will be copied back to Login Node(GPFS) once they are generated.
>> Here we got an old problem, all IO nodes are trying
>> to write files in the same directory, which we are trying to avoid all the
>> way.
>> My solution would be modify the ssh-provider source code, implement an
>> asynchronous collector logic there.
>>     
>
> Can you describe what is going on here more explicitly.
>
> How do multiple IO nodes end up writing to the same GPFS directory?
>   
In previous case, we have 512 IO nodes each create 1 file in the same 
directory, that would take 30 minutes to finish.
Besides, some time only 510 files could be created.
> It is unclear to me from what you write how that comes about - as I 
> understand it:
>
>  . submit side data files are posix-accessed only by the swift submit-side 
> client
>   
yes
>  . files on the I/O nodes (the remote sites) use pset-local storage
>   
yes
>  . any communication between the I/O nodes and submit-side client happens 
> over ssh.
>   
yes
> Where does an I/O node access machine-wide GPFS?
>   
data transfer from  I/O nodes to submit-side client is  writing to GPFS 
through ssh-provider.

zhao