[Swift-user] Virtual data schema / catalog?

[Ben Clifford]

There's a long term plan to have something like a VDC but no 
implementation.

Tibi has worked on an experiment management database for one specific 
project - maybe he will comment on how that overlaps here.

The origins of the XML and KML files are roughly analogous to VDS1's XML 
form of VDL and to Condor DAGman graphs (respectively).

The XML files are more or less the same description as the SwiftScript 
.swift files, but in an XML form. The KML files are lower-level, 
describing various things that need to happen inside the execution 
environemnt itself.

The intention (or at least my intention) was/is that the XML would be 
stuff that would go into a VDC, with the KML files not being 
standardised/shareable.



In addition, there's a utility called kickstart which will generate 
invocation information for runs on the actual sites that the jobs run on. 
Swift can be configured to always run that and bring the files back to
the submitting system.

kickstart was also present in VDS1 - its the same executable for both.

The invocation records from this are also in XML form.


So the short answer is: its easy to get a bunch of XML descriptions of 
both the high level workflow and actual invocations dumped into files in a 
directory. We haven't got anything in Swift that will do anything with 
those files.

If I was hacking round with this, then given the XML nature of the 
invocation records and the XML intermediate for of SwiftScript, I'd be 
inclined to make something that would import them all into an XML database 
like Xindice and then play around making XPath queries against that.

--