[Swift-devel] How does swift know if a task is successful

[Ben Clifford]

So if Swift could remove the dependency between staging out and starting 
subsequent jobs (a subset of what has been talked about before), would you 
still need to hack out the stageout code?

> To solve this problem, we built a P2P data network on BGP over torus 
> network. So the basic logic for this is that if a wrapper.sh found a 
> piece of intermediate data, it registered this data with (name, rank of 
> the CN) to a Centralized Hash Table(CHT). Next time, when a job needs 
> this data, first it looks this data up in CHT, gets the rank of the 
> remote node, convert the RANK to IP, fetch the data directly.

When we talked in December, I think this bit was done with posix 
filesystem access. But it sounds like you are doing something different 
now.

I've looked at abstracting that worker<->site shared filesystem code in 
the past (and have some patches floating round in half-written state) - 
can you send me your modified wrapper.sh so I can see how you do things?

--