[Swift-devel] [Bug 76] disable intermediate stageout of data

[Mihael Hategan]

On Thu, 2007-07-05 at 14:05 +0000, Ben Clifford wrote:
> how does it know where files are now, between jobs?

That's the thing. They're always on localhost.

> 
> On Thu, 5 Jul 2007, Mihael Hategan wrote:
> 
> > I think you're missing something. You need to remember where the files
> > are. The mapping information becomes insufficient. It tells you where
> > some initial files were, but it won't contain any site information. And
> > that's good, because the decision of where something is done is made at
> > run-time. But you still need some store (even though probably
> > memory-based and only persistent through one swift run).
> > 
> > On Thu, 2007-07-05 at 04:41 +0000, Ben Clifford wrote:
> > > I don't think that's true.
> > > 
> > > If data files are labelled with URIs rather than 
> > > paths-relative-to-submit-directory, then those URIs are understandable 
> > > without a VDC-as-entity.
> > > 
> > > You don't need a separate VDC to tell you how to get at myfile here:
> > > 
> > >   file myfile <"gsiftp://terminable.ci.uchicago.edu/scratch/foo/">;
> > > 
> > > The 'data file pointer store' exists already - its the hierarchical 
> > > namespace that is rooted in IANA's management of the URI and DNS space, 
> > > continues to UC's management of DNS space and then down to my management 
> > > of terminable's filesystem space and then down to whoever owns the foo 
> > > directory.
> > > 
> > > 
> > > On Sun, 1 Jul 2007, bugzilla-daemon at mcs.anl.gov wrote:
> > > 
> > > > http://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=76
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > ------- Comment #1 from hategan at mcs.anl.gov  2007-07-01 02:18 -------
> > > > This would require a data file pointer store (VDC like thing) which can record
> > > > where intermediate files are instead of assuming they are always available on
> > > > the submit host.
> > > > 
> > > > 
> > > > 
> > > _______________________________________________
> > > Swift-devel mailing list
> > > Swift-devel at ci.uchicago.edu
> > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > 
> > 
> > 
>