[Swift-user] Swift restarts with iterate?

[Mihael Hategan]

On Wed, 2009-04-15 at 19:02 +0000, Ben Clifford wrote:
> On Wed, 15 Apr 2009, Michael Wilde wrote:
> 
> > Is the restart feature designed to correctly handle restarts of scripts with
> > active, possibly nested, iterate statements?
> 
> There was no intention that such would not work.
> 
> > The use case of interest here is to run a single copy of swift continuously,
> > or for extended periods, doing a task graph of work, sleeping, and repeating
> > indefinitely.
> 
> I've considered that as a way of handling something like streaming 
> datasets.
> 
> Doing that should work in as much as it should accomodate new data 
> appearing.
> 
> However I'm unsure of the memory usage scalability compared to a run where 
> you had all the data in place at the start of a single run - Swift will 
> still make karajan threads to attempt (and then optimise away) already 
> done executions, and will still have an in-memory representation of each 
> data object already processed.

While thinking of the scalability issues a while ago when I did the
foreach limiting, I concluded that a solution to that may exist.

Currently we use a certain scheme to detect when a piece of data stops
being written to, such that it can be considered "closed".

Similarly, it may be possible to determine when a piece of data will not
be referenced any more, and consequently remove in-memory references to
it and its associated data structures.