[Swift-devel] [Bug 76] disable intermediate stageout of data

[Mihael Hategan]

On Thu, 2007-07-05 at 16:25 +0000, Ben Clifford wrote:
> they're always in the place that the path name says they are. whether its 
> a URI or a local relative path.

Right, but whereas in the current scheme you can assume the site is
localhost, because files are always staged back to localhost, if you
don't do the stage-out, that assumption goes away. In that case, the
site information needs to be recorded.

> 
> On Thu, 5 Jul 2007, Mihael Hategan wrote:
> 
> > On Thu, 2007-07-05 at 14:05 +0000, Ben Clifford wrote:
> > > how does it know where files are now, between jobs?
> > 
> > That's the thing. They're always on localhost.
> > 
> > > 
> > > On Thu, 5 Jul 2007, Mihael Hategan wrote:
> > > 
> > > > I think you're missing something. You need to remember where the files
> > > > are. The mapping information becomes insufficient. It tells you where
> > > > some initial files were, but it won't contain any site information. And
> > > > that's good, because the decision of where something is done is made at
> > > > run-time. But you still need some store (even though probably
> > > > memory-based and only persistent through one swift run).
> > > > 
> > > > On Thu, 2007-07-05 at 04:41 +0000, Ben Clifford wrote:
> > > > > I don't think that's true.
> > > > > 
> > > > > If data files are labelled with URIs rather than 
> > > > > paths-relative-to-submit-directory, then those URIs are understandable 
> > > > > without a VDC-as-entity.
> > > > > 
> > > > > You don't need a separate VDC to tell you how to get at myfile here:
> > > > > 
> > > > >   file myfile <"gsiftp://terminable.ci.uchicago.edu/scratch/foo/">;
> > > > > 
> > > > > The 'data file pointer store' exists already - its the hierarchical 
> > > > > namespace that is rooted in IANA's management of the URI and DNS space, 
> > > > > continues to UC's management of DNS space and then down to my management 
> > > > > of terminable's filesystem space and then down to whoever owns the foo 
> > > > > directory.
> > > > > 
> > > > > 
> > > > > On Sun, 1 Jul 2007, bugzilla-daemon at mcs.anl.gov wrote:
> > > > > 
> > > > > > http://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=76
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > ------- Comment #1 from hategan at mcs.anl.gov  2007-07-01 02:18 -------
> > > > > > This would require a data file pointer store (VDC like thing) which can record
> > > > > > where intermediate files are instead of assuming they are always available on
> > > > > > the submit host.
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > > 
> > > > 
> > > > 
> > > 
> > 
> > 
>