[Swift-devel] [Bug 76] disable intermediate stageout of data

[Ben Clifford]

right.

On Thu, 5 Jul 2007, Mihael Hategan wrote:

> On Thu, 2007-07-05 at 16:25 +0000, Ben Clifford wrote:
> > they're always in the place that the path name says they are. whether its 
> > a URI or a local relative path.
> 
> Right, but whereas in the current scheme you can assume the site is
> localhost, because files are always staged back to localhost, if you
> don't do the stage-out, that assumption goes away. In that case, the
> site information needs to be recorded.
> 
> > 
> > On Thu, 5 Jul 2007, Mihael Hategan wrote:
> > 
> > > On Thu, 2007-07-05 at 14:05 +0000, Ben Clifford wrote:
> > > > how does it know where files are now, between jobs?
> > > 
> > > That's the thing. They're always on localhost.
> > > 
> > > > 
> > > > On Thu, 5 Jul 2007, Mihael Hategan wrote:
> > > > 
> > > > > I think you're missing something. You need to remember where the files
> > > > > are. The mapping information becomes insufficient. It tells you where
> > > > > some initial files were, but it won't contain any site information. And
> > > > > that's good, because the decision of where something is done is made at
> > > > > run-time. But you still need some store (even though probably
> > > > > memory-based and only persistent through one swift run).
> > > > > 
> > > > > On Thu, 2007-07-05 at 04:41 +0000, Ben Clifford wrote:
> > > > > > I don't think that's true.
> > > > > > 
> > > > > > If data files are labelled with URIs rather than 
> > > > > > paths-relative-to-submit-directory, then those URIs are understandable 
> > > > > > without a VDC-as-entity.
> > > > > > 
> > > > > > You don't need a separate VDC to tell you how to get at myfile here:
> > > > > > 
> > > > > >   file myfile <"gsiftp://terminable.ci.uchicago.edu/scratch/foo/">;
> > > > > > 
> > > > > > The 'data file pointer store' exists already - its the hierarchical 
> > > > > > namespace that is rooted in IANA's management of the URI and DNS space, 
> > > > > > continues to UC's management of DNS space and then down to my management 
> > > > > > of terminable's filesystem space and then down to whoever owns the foo 
> > > > > > directory.
> > > > > > 
> > > > > > 
> > > > > > On Sun, 1 Jul 2007, bugzilla-daemon at mcs.anl.gov wrote:
> > > > > > 
> > > > > > > http://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=76
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > ------- Comment #1 from hategan at mcs.anl.gov  2007-07-01 02:18 -------
> > > > > > > This would require a data file pointer store (VDC like thing) which can record
> > > > > > > where intermediate files are instead of assuming they are always available on
> > > > > > > the submit host.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > _______________________________________________
> > > > > > Swift-devel mailing list
> > > > > > Swift-devel at ci.uchicago.edu
> > > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > > > 
> > > > > 
> > > > > 
> > > > 
> > > 
> > > 
> > 
> 
>