[Swift-user] Running Python scripts on Swift

[Michael Wilde]

> I have a few python scripts that chain input/output, and I was
> wondering what the best way to call these scripts?

Can you clarify? Do you want to have Swift chain them, or chain them within a single shell script?  Either approach is possible and reasonable.

Chain them in Swift if:
- there's a modularity benefit to keeping them separate
- you might want to run them on different resources (sites)

Chain them in a shell (or similar) script if:
- they will never need to be run separately
- their run time is so short that its better to pass the data within a shell script on local filesystems or env vars, etc.

> Thus far, I've tried placing them into a shell script, but I'm not
> sure how to make the shell scripts take arguments. How would one do
> this?

I think you're just missing something like the shell's "-c" argument.

You should test your calling conventions from a shell command line first; then make sure that your Swift script is generating the correct command line from your app() function specification.

In your example below, where you are constructing the shell script in a file, just remove the word "source".

> app (reducefile sout) runmapreduce(shellfile shellscript, inputfile f,
> pythonfile map, pythonfile combine, pythonfile reduce){
> shell "source" @shellscript @f @map @combine @reduce
> stdout=@filename(sout);
> }
> shellfile
> mapreduce_shell_script<single_file_mapper;file="mapreduce.sh">;

Omit the argument "source" since your next argument is an actual shell script.

Assuming "shell" is a tc.data entry pointing to /bin/sh etc, the shell wont take "source" as a command unless its preceded by "-c". But then it wants a single string as an argument.

There's a few subtleties you need to work out between Swift, its qouting and command-line construction conventions, and the shell youre using, and theres a few different ways to do this.

We need to document these approaches, but for now, with a bit of experimentation its pretty easy to make such shell calling conventions work very reasonably.

- Mike