[Swift-devel] GPFS issue of SWIFT on BGP

[Zhao Zhang]

Hi, All

I am working with Allan on applying CIO to SWIFT on BGP, now we are 
blocked by a ssh-provider issue.
Here is the description: we made ssh-provider working as the data 
provider, and I tested it with multiple psets, it is working fine.
Login Node ----- submit host
IO Node -------- remote site
Compute Node -- workers

Now, we start swift on Login Node, and the working directory will be 
created on IO Node, so that all intermediate files and final
result files will be copied back to Login Node(GPFS) once they are 
generated. Here we got an old problem, all IO nodes are trying
to write files in the same directory, which we are trying to avoid all 
the way.

My solution would be modify the ssh-provider source code, implement an 
asynchronous collector logic there.

Do you have any other ideas about this issue? Or other alternative 
design? Thanks so much.

best wishes
zhangzhao