[Swift-user] Swift restarts with iterate?
hategan at mcs.anl.gov
Wed Apr 15 14:12:58 CDT 2009
On Wed, 2009-04-15 at 19:02 +0000, Ben Clifford wrote:
> On Wed, 15 Apr 2009, Michael Wilde wrote:
> > Is the restart feature designed to correctly handle restarts of scripts with
> > active, possibly nested, iterate statements?
> There was no intention that such would not work.
> > The use case of interest here is to run a single copy of swift continuously,
> > or for extended periods, doing a task graph of work, sleeping, and repeating
> > indefinitely.
> I've considered that as a way of handling something like streaming
> Doing that should work in as much as it should accomodate new data
> However I'm unsure of the memory usage scalability compared to a run where
> you had all the data in place at the start of a single run - Swift will
> still make karajan threads to attempt (and then optimise away) already
> done executions, and will still have an in-memory representation of each
> data object already processed.
While thinking of the scalability issues a while ago when I did the
foreach limiting, I concluded that a solution to that may exist.
Currently we use a certain scheme to detect when a piece of data stops
being written to, such that it can be considered "closed".
Similarly, it may be possible to determine when a piece of data will not
be referenced any more, and consequently remove in-memory references to
it and its associated data structures.
More information about the Swift-user