[Swift-devel] Re: Another performance comparison of DOCK

[Zhao Zhang]

Hi, Ben

I got a log file of 6084 successful runs on BGP. Check it here, 
terninable:/home/zzhang/swift_file/dock2-20080412-1609-99cy0z4g.log

zhao

Michael Wilde wrote:
> Ben, can you confirm this: to turn off all job submission throttling 
> (for Falkon), the correct setting for each of the following props is 
> "off"?
>
> # remove all limits on job submit rates
> throttle.submit=off
> throttle.host.submit=off
> throttle.score.job.factor=off
>
> Long ago (circa Nov) it seemed "off" didn't give the the wide-open 
> throttle effect I was looking for, but "off" is a more clear setting 
> than "big numbers" if we know it to be working as expected.
>
> - Mike
>
>
>
> On 4/12/08 5:22 AM, Ben Clifford wrote:
>> On Sat, 12 Apr 2008, Zhao Zhang wrote:
>>
>>>   i) what files are shared (so swift will only stage-in once)
>>>        how many files per job and how big in total per job?
>>>  
>>> I am not familiar with definition of "stage-in". Mike, could you 
>>> help to
>>> explain this?
>>
>> When a job runs in Swift, it looks like this:
>>
>>     stage in the input files - copy the input files from where they are
>>        stored to where they are needed for execution
>>     execute (using falkon in this case)
>>     stage out the output files
>>
>> The execute stage is what falkon is involved in; there are other 
>> mechanisms to move files around into the appropraite
>>
>> So really I want to know what are the input files to your jobs.
>>
>>> on GPFS of Blue Gene
>>
>> If everything is on the same filesystem on bluegene, an interesting 
>> idea springs to mind of potentially only symlinking the input files 
>> rather than copying them around.
>>
>> I can have a play with that next week perhaps.
>>
>