[Swift-devel] discussion for swift output file path

[Zhao Zhang]

Ben Clifford wrote:
> On Thu, 6 Nov 2008, Zhao Zhang wrote:
>
>   
>> I am working on integrate the Collective IO system and swift on BGP. Before
>> that, for the purpose of put swift into production work,
>>     
>
> It would be useful if the changes, if they're useful, end up in the 
> production Swift SVN - keeping everyone's codebase fairly tightly 
> converged seems a good thing to me.
>
> Points 1 and 3 below seem more difficult, but point 2:
>
>   
>> 2. change wrapper.sh: copy files from local ramdisck to GPFS using dd           
>> instead of cp,      straghtforward.                     
>>     
>
> is easy to put in. Is there a measurable performance increase using dd 
> rather than cp? And dd using default block size or different ones?
>   
yep, I made this working on surveyor. cp on CN uses a lined-buffer for 
output, thus a bad performance. the suggested block size of dd from 
Kamil is 128KB.

zhao
>   
>> 1. change vdl-int.k: create hierarchical directory in shared/,
>> straightforward
>>     
>
>   
>> 3. change somewhere in swift to make swift know where the data is, the path of
>> the output file in jobdir/shared/    unknown
>>     
>
>