[Swift-devel] turning off Swift's data caching

[Tim Armstrong]

Be warned that if you go down the path of having Swift/T file variables
based on changing file-system files, then the language doesn't really
guarantee any particular behaviour: the behaviour may be surprising.  You
might be able to get it to do what you want for now: as Justin mentioned,
there isn't runtime caching of files at the moment (this of course may
change in future).  Even in current versions, the optimizer will make the
assumption that file contents don't change.  I.e. that files are
single-assignment.

Justin's specific example works in current versions of Swift/T, but I won't
guarantee that it will work in future versions as the optimizer gets
smarter.  There are also subtle changes you can make to the code that will
make it cache the results of compute().  This is because, even though it
doesn't cache things at runtime, compiler optimizations will assume that
the contents of files don't change.  In general, the Swift model has to
assume that file variables are unchanging, in part because its declarative,
but also because we don't really have a strong notion of sequencing or
time, so it's not really possible to provide any sort of reasonable
semantics to a programmer for mutating files, especially when performance
is a concern.

I spoke to Justin about this off-list and it sounds like he has a
workaround in mind that involves having a sequence number argument to
compute() that should be more robust.  I did want to send this reply to
this list so that others know that "here be dragons" when it comes to
dealing with mutating files in Swift/T as well as /K.

- Tim


On Mon, Apr 22, 2013 at 1:40 PM, Justin M Wozniak <wozniak at mcs.anl.gov>wrote:

>
> Normally, Swift variables are write-once.  Swift assumes the contents of
> a file are constant and is allowed to cache.
>
> However, you can do this in Swift/T.  Swift/T does not stage files, so
> there is no cache, and there are new constructs for handling external
> data dependencies.
>
> The following script launches compute() every 10 seconds if data.txt has
> changed.  Touch done.txt to exit.
>
> import io;
> import files;
> import sys;
>
> app compute(file f) {
>    "/bin/echo" "compute on:" f;
> }
>
> (int o) check_done(int i) {
>    i =>
>    if (file_exists("done.txt")) { o = 1; }
>    else                         { o = 0; }
> }
>
> global const int interval = 10;
>
> main {
>    int i, done;
>    for (i = 0, done = 0; done == 0; i = next_i, done = check_done(i)) {
>      int next_i;
>      i => {
>        if (file_mtime("data.txt") > clock_seconds()-interval) {
>          compute(input_file("data.txt"));
>        }
>        sleep(itof(interval)) => next_i = 1;
>      }
>    }
> }
>
>
> On 04/21/2013 12:26 AM, Ketan Maheshwari wrote:
> > Hi,
> >
> > I am trying to mimic a scenario where an external process is streaming
> > data into a file which is being captured at same or lesser frequency
> > by updating a file periodically which is then cat'ed by Swift into an
> > output file.
>
> --
> Justin M Wozniak
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20130422/2e602433/attachment.html>