[Swift-user] Reduction trees

[Mihael Hategan]

On Mon, 2014-05-19 at 14:54 -0700, Mihael Hategan wrote:
> Well, ... I can probably write a hackish split function that you can use
> with 0.94. Give me a few hours.

Ok, here's the basic idea:

(file[][] result) split(file[] array, int n) {
  // need to explicitly keep track of keys since
  // writeData does not write the array keys
  int[auto] keys;
  foreach v, k in array {
    keys << k;
  }
  
  file kf = writeData(keys);
  
  file kfs = splitArrayApp(kf, n); // make pointers to original keys
  
  int[][] skeys = readStructured(kfs);
  
  foreach slice, sliceIndex in skeys {
    foreach index in slice {
      result[sliceIndex][index] = array[index];
    }
  }
}

Attached are full versions of scripts. You will need to add
splitArrayApp to tc.data. Please try to run it and let me know if it
works. My development 0.94 might be slightly different from what you
have.

Mihael
-------------- next part --------------
type file;

// ----- Start of split related stuff

app (file outf) splitArrayApp(file inf, int n) {
	splitArrayApp @filename(inf) n stdout=@filename(outf);
}

(file[][] result) split(file[] array, int n) {
	// need to explicitly keep track of keys since
	// writeData does not write the array keys
	int[auto] keys;
	foreach v, k in array {
		keys << k;
	}
	
	file kf = writeData(keys);
	
	file kfs = splitArrayApp(kf, n); // make pointers to original keys
	
	int[][] skeys = readStructured(kfs);
	
	foreach slice, sliceIndex in skeys {
		foreach index in slice {
			result[sliceIndex][index] = array[index];
		}
	}
}

// ----- End of split related stuff

file[] array;

app (file outf) gen(int i) {
	echo i stdout=@filename(outf);
}

foreach i in [1:13] {
	file f <single_file_mapper; file=@strcat("f", i * 2)>;
	f = gen(i * 3);
	array[i * 2] = f;
}

file[][] split = split(array, 5);

foreach v1, k1 in split {
	foreach v2, k2 in v1 {
		tracef("array[%i][%i] -> %s\n", k1, k2, @filename(v2));
	} 
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: splitArray
Type: text/x-python
Size: 297 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20140519/479ddf3d/attachment.py>