[Swift-devel] problems with external dependencies

[Michael Wilde]

On 3/24/09 4:30 AM, Ben Clifford wrote:
> On Mon, 23 Mar 2009, Michael Wilde wrote:
> 
>> out = ls("datadir/", sync);
> 
> This ls needs an absolute path - when it runs, it is in its own jobdir 
> which (deliberately) has none of the other files staged in.

Yes, makes perfect sense. I did exactly that in the production code, but 
neglected to do it in this test code, and was fooled by the answer. I 
understand this part completely.

I then need to correct this and go back and experiment more with the 
synchronization semantics, mainly related to array closing and procedure 
entry/exit semantics. Despite the fact that I can use these pretty 
effectively, I still dont have a solid, comfortable mental model of how 
some of thee aspects work.

Back to the main point: I need to dissect this example to better 
understand how it lets me wait for all the apps to finish without 
putting an external var on each one, and hence avoids the array of 
externals. This violates my (most recent) understanding of procedure 
return, in that generate() would return to the calls *before* all the 
ls() procs called from its foreach loop returns. That seems to be the 
crux of the matter.

I need to go back through this thread, but I though Mihael confirmed 
that this return would indeed happen. I also need to look at what I did 
wrong in my prior tests on this, as I thought I tried this approach 
first. Maybe the full-path mistake misled me from the start. Very subtle 
stuff.

> You need to specify the absolute path to your data directory there.
> 
> One way to think of it is: rather than calling the directory 'sync', you 
> coul call it 'myDataDir' - that external variable represents the external 
> directory being created and passed around. But, because you have declared 
> that data 'external' that means Swift will never attempt to put you in the 
> right directory for access to that data or prepare any input files for you 
> or anything like that (indeed, your external data might not even be file 
> based - this was originally implemented for someone who wanted to do 
> database access, I think)

Yes, that part is clear, and was clear from the start. I just goofed in 
this test. In the production oops code, I pass in "cwd" as an arg and 
prepend that to the "outdir" path, which is also an arg.

Btw, oops is in svn at:
https://svn.ci.uchicago.edu/svn/oops
the swift code and supporting scripts (mappers etc) are in there under 
swift/