[Swift-devel] [Bug 76] disable intermediate stageout of data

[Ben Clifford]

how does it know where files are now, between jobs?

On Thu, 5 Jul 2007, Mihael Hategan wrote:

> I think you're missing something. You need to remember where the files
> are. The mapping information becomes insufficient. It tells you where
> some initial files were, but it won't contain any site information. And
> that's good, because the decision of where something is done is made at
> run-time. But you still need some store (even though probably
> memory-based and only persistent through one swift run).
> 
> On Thu, 2007-07-05 at 04:41 +0000, Ben Clifford wrote:
> > I don't think that's true.
> > 
> > If data files are labelled with URIs rather than 
> > paths-relative-to-submit-directory, then those URIs are understandable 
> > without a VDC-as-entity.
> > 
> > You don't need a separate VDC to tell you how to get at myfile here:
> > 
> >   file myfile <"gsiftp://terminable.ci.uchicago.edu/scratch/foo/">;
> > 
> > The 'data file pointer store' exists already - its the hierarchical 
> > namespace that is rooted in IANA's management of the URI and DNS space, 
> > continues to UC's management of DNS space and then down to my management 
> > of terminable's filesystem space and then down to whoever owns the foo 
> > directory.
> > 
> > 
> > On Sun, 1 Jul 2007, bugzilla-daemon at mcs.anl.gov wrote:
> > 
> > > http://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=76
> > > 
> > > 
> > > 
> > > 
> > > 
> > > ------- Comment #1 from hategan at mcs.anl.gov  2007-07-01 02:18 -------
> > > This would require a data file pointer store (VDC like thing) which can record
> > > where intermediate files are instead of assuming they are always available on
> > > the submit host.
> > > 
> > > 
> > > 
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > 
> 
>