[Swift-devel] Format of site tests

[Michael Wilde]

Sarah,

Justin and I use a format for site testing that is roughly as shown below: a single script that emits swift.property settings, tc, sites.xml, the swift script.

It can and should create any input data and external mappers as needed.

I hope this helps you get started creating and running these tests. You and Justin should schedule a call to discuss how to integrate these tests into the test suite.

This will take some discussion, which we should do on this list, but hopefully this is a good starting point for the site tests.

I think Ben has something similar in the existing tests, but I have not looked at those yet.

- Mike

[mwilde at master tests]$ cat t1.sh
cat >tc <<EOF

sge cat /bin/cat null null null

EOF

cat >sites.xml <<EOF
<config>
  <pool handle="sge">
    <execution provider="sge" url="none" />
    <profile namespace="globus" key="pe">shm</profile>
    <profile namespace="globus" key="maxwalltime">00:01:00</profile>
    <profile namespace="karajan" key="initialScore">10000</profile>
    <profile namespace="karajan" key="jobThrottle">.20</profile>
    <filesystem provider="local" />
    <workdirectory>$PWD</workdirectory>
  </pool>
</config>

EOF

cat >cf <<EOF

wrapperlog.always.transfer=true
sitedir.keep=true
execution.retries=0
lazy.errors=false
status.mode=provider
use.provider.staging=false
provider.staging.pin.swiftfiles=false

EOF

cat >catsn.swift <<EOF

type file;

app (file o) cat (file i)
{
  cat @i stdout=@o;
}

file out[]<simple_mapper; location="outdir", prefix="f.",suffix=".out">;
foreach j in [1:@toint(@arg("n","1"))] {
  file data<"data.txt">;
  out[j] = cat(data);
}

EOF

echo Hi There Swift! >data.txt

swift -config cf -tc.file tc -sites.file sites.xml catsn.swift -n=100


-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory