[Swift-devel] discussion for swift output file path

[Mihael Hategan]

On Fri, 2008-11-07 at 01:07 +0000, Ben Clifford wrote:
> On Thu, 6 Nov 2008, Mihael Hategan wrote:
> 
> > It may also make sense to coalesce all the trees into one, such that
> > data, info, and temporary job dirs are created in the same directory
> > (e.g. (info/a/0, job/a/0) becomes (a/0/info, a/0/job)), since it will
> > reduce the overall number of operations.
> 
> Perhaps does for info and job where the access/lock patterns are the same 
> under both the info and job tree.
> 
> But the shared data tree needs to be hashed by something based on the file 
> (eg a hash of the filename) rather than on something specific to a 
> particular job.
> 
> Putting the shared directories under the same hash tree as info/ and job/ 
> would reduce the number of directories in total but would increase lock 
> contention on those directories, because cached data file accesses follow 
> a different pattern from info/ and job/ accesses.

Only if we assume non-reentrant read locks. And this would need some
investigation, but if concurrent reads do not create contention, then
there is no problem, because once a job is done, there is no more
writing to its directory.