[Swift-devel] discussion for swift output file path

[Zhao Zhang]

Hi, All

I am working on integrate the Collective IO system and swift on BGP. 
Before that, for the purpose of put swift into production work,
we need to change the output file path. For now, wrapper.sh would copy 
all output files to jobdir/shared/, on BGP, all output files will
be written to one directory, which I am sure will cause the GPFS lock 
mechanism, thus introduce unacceptable latency.

So the easiest way to fix this is "make a hierarchical directory in 
shared/ and we already did in info/ and jobs/". Several changes we need:
     
place-need-change                                                        
                                             diffculty
1. change vdl-int.k: create hierarchical directory in shared/,         
                                     straightforward
2. change wrapper.sh: copy files from local ramdisck to GPFS using dd 
instead of cp,      straghtforward.
3. change somewhere in swift to make swift know where the data is, the 
path of the output file in jobdir/shared/    unknown

Any comments will be appreciated.

best wishes
zhangzhao