[Swift-user] Virtual data schema / catalog?

[Ben Clifford]

> Being able to look through a provenance chain would be good because it
> could allow selective regeneration of data sets.  I.e. in some cases I
> don't want to run the entire workflow, I just want software that will
> figure out which datasets in a workflow are missing, and then recompute
> only those pieces (and any others that depend on them).

Also related to this, then:

Swift (or rather the Karajan workflow engine underneath it) has this 
concept of restart logs. These are implemented at a lower level than the 
XML.

Briefly, if you have a KML file, you can run part of it, have a failure, 
let the system abort and write out a restart log; and then you can run 
again using the restart log to ignore work already done.

Because this happens at the KML level, its suspect its not something that 
you can really put in a database and come back to with a different 
(version of the) workflow - its more intended for "i set this day long 
workflow running; it died overnight; tomorrow I will restart it".

There's a brief section on this in the tutorial, at
http://www.ci.uchicago.edu/swift/guides/tutorial.php#id2860757

"16. Starting and restarting"

--