[Swift-devel] Interesting observation when running Swift

[Tiberiu Stef-Praun]

Interesting.

Does anyone else think that monitoring the filesystem could be a useful idea ?

For instance it could help with file-driven dependencies, in scenarios
where we want to have continuous workflows, or compose independent
wokflows. The filesystem would act as the publish-subscribe mechanism
for some workflow cases.

Tibi

On 4/10/07, Mihael Hategan <hategan at mcs.anl.gov> wrote:
> Swift doesn't monitor the file system.
> Data driven doesn't mean that it does magic in the background. It means
> that you have to express data dependencies in the code.
>
> On Tue, 2007-04-10 at 11:47 -0500, Tiberiu Stef-Praun wrote:
> > I have a workflow along these lines:
> >
> > // this one generates outputsStage1[]
> > stage1()
> > // this one merges the stage1 outputs
> > stage2(outputsStage1[])
> >
> > note that it is not outputsStage1=stage1()
> >
> > Since the outputsStage1 files were not generated yet, I expected that
> > Karajan waited for them to be created before running stage2, but that
> > was not the case: stage2 was executed when the workflow started (and
> > it failed) and caused the workflow to fail.
> >
> > I know how to fix the workflow, that is not the issue. The issue is
> > that I expected the workflow to be data-driven, but it seems to be
> > code driven. Explanation: it attempted to execute a section even if
> > its input files were not available.
> >
> > Correct me if I am wrong.
> > Tibi
> >
>
>


-- 
Tiberiu (Tibi) Stef-Praun, PhD
Research Staff, Computation Institute
5640 S. Ellis Ave, #405
University of Chicago
http://www-unix.mcs.anl.gov/~tiberius/