[Swift-user] Reduction trees

[Mihael Hategan]

Hi,

Can you be more specific about what you mean by "subsets of keys" below?
Specifically, how are these sub sets defined?

Mihael

On Sun, 2014-05-18 at 22:16 +0000, Bronevetsky, Greg wrote:
> I need to implement a reduction three in Swift to aggregate the
> results of many individual runs. I've written the simple algorithm
> below, which works when all my runs correspond to numbers in a fixed
> range. However, in reality I have a regular or associative array of
> files produced by individual runs.  How do I adapt the code below to
> this scenario? I don't see any way to iterate over subsets of array
> indexes/keys. Thanks!
> 
> Greg Bronevetsky
> Lawrence Livermore National Lab
> (925) 424-5756
> bronevetsky at llnl.gov<mailto:bronevetsky at llnl.gov>
> http://greg.bronevetsky.com
> 
> 
> type file;
> 
> app (file outFile) gen(string arg)
> {
>   echo arg stdout=@filename(outFile);
> }
> 
> app (file summaryFile) catBase(file inFiles[]) {
>   cat @filenames(inFiles) stdout=@filename(summaryFile);
> }
> 
> // Concatenate the text of numbers in range [minR - maxR) (includes minR, not maxR)
> (file summaryFile) catTree(int minR, int maxR, int radix, int level) {
>   file subFiles[];
>  if((maxR-minR) <= radix) {
>     //tracef("catTree: leaf: minR=%i, minR=%i\n", minR, maxR);
>     foreach b in [0: (maxR-minR)-1] {
>       //tracef("catTree: leaf: b=%i\n", b);
>       file curLeafFile <single_file_mapper; file=@strcat("file.", at toString(minR+b))>;
>       (curLeafFile) = gen(@toString(minR+b));
>       subFiles[b] = curLeafFile;
>     }
>   } else {
>     int size=maxR-minR;
>     //tracef("catTree: node: minR=%i, minR=%i\n", minR, maxR);
>     foreach b in [0: radix-1] {
>       file curNodeFile <single_file_mapper; file=@strcat("node.level_", at toString(level),".startVal_", at toString(minR+b))>;
>       int start = minR + (size*b)%/radix;
>       int end   = minR + (size*(b+1))%/radix;
>       //tracef("catTree: node: b=%i [%i - %i]\n", b, start, end);
>       (curNodeFile) = catTree(start, end, radix, level+1);
>       subFiles[b] = curNodeFile;
>     }
>   }
>   //tracef("catBase: inFiles=%q\n", @filenames(subFiles));
>   (summaryFile) = catBase(subFiles);
> }
> 
> 
> file allFile <single_file_mapper; file="all">;
> (allFile) = catTree(0, 31, 2, 0);
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user