[Swift-devel] discussion for swift output file path

[Mihael Hategan]

On Fri, 2008-11-07 at 01:23 +0000, Ben Clifford wrote:
> On Thu, 6 Nov 2008, Mihael Hategan wrote:
> 
> > Only if we assume non-reentrant read locks. And this would need some
> > investigation, but if concurrent reads do not create contention, then
> > there is no problem, because once a job is done, there is no more
> > writing to its directory.
> 
> If job output files are left in their job directory, rather than put in 
> some hashed-by-filename shared directory, I think that is true. Otherwise, 
> the access patterns for any given hash bucket for a hash-on-filename are 
> going to be fairly random, with reads and writes happening at all sorts of 
> different times and from many different places.
> 
> Keeping output files in job-specific directories seems a reasonable thing 
> to do performance wise, but means more awareness on the submit side about 
> which files came from which job.

We already have awareness of which file on which site. I don't think
this is any different.