[Swift-devel] Swift data distribution strategy

[Mihael Hategan]

----- Zhao Zhang <zhaozhang at uchicago.edu> wrote:
> Hi,
> 
> I got a problem when I do a 512-compute node scale test.
> There are 15351 input files for this computation, and there are 8 sites.
> The question is: when I start swift, are all 15351 input files be copied 
> to each of the 8 sites?
> 
> By by test, it is yes.

It's more like "no" actually. Swift first selects a site for each job 
that can be run. After that, it stages each job's files to the site 
that was selected for that job.

> Does swift has an option so that only on demand 
> input files are copied?
> 
> best wishes
> zhangzhao
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel