[Swift-devel] Iterative PageRank in Swift

[Michael Wilde]

> ... In my partition app definition,
> app (file t[]) partition (file f) {
>     partition @filename(f) "16";
> }
> 
> I did not put output files as parameter. I think that is why it does
> not work right.
> The command executes as:
> bin/partition.py input_file 16
> 
> and it produces 16 output files with the naming convention of
> input_file-0, input-file-1, ..., input-file-15

That seems close to OK as-is.  The output files are OK as a parameter to the swift app function partition(). This will cause Swift to look for that array of files and pass them to any downstream functions in the data flow that depend on them.  They don't need to be mentioned on the partition *command line* so long as partition.py correctly creates these 16 files.

The first problem I would look for here is whether partition will do so correctly if the input path name has directory components in it.  In other words, given this command line:

  bin/partition.py d1/d2/input-file 16

it should create ./d1/d2/input-file-0 ... ./d1/d2/input-file-15, NOT ./input-file-16

You can check if its doing this both standalone, and under Swift by doing a find on the job dir in the workdirectory.

- Mike