[Swift-user] Advice on mapping input arguments

[TJ Lane]

Swift Users,

I am wondering if I could get some advice on the best way to do the
following in Swift: I want to run a series of simulations performing a
parameter scan, for each parameter combination farming the work out to
clusters I have access to here at Stanford, and collect the results back
onto my desktop.

I've gotten some minimal working examples of swift up and running, but hit
a roadblock on something quite simple: what's the best way to pass a large
number of parameters into a swift script? I have a big list of parameter
combinations I'd like to run, and am searching for a sane way to pass all
of these into my swift app call.

Originally, I thought I'd be able to use the CSV mapper to pass a bunch of
arguments from a CSV file into swift -- it seemed perfect! As a bonus, I
hoped the CSV file would act as a record of my work, namely what parameters
were used to generate what file. But it seems that the CSV mapper
automatically maps the entries in the CSV file to swift "mapper" objects --
i.e., it expects my CSV data fields are all files, where as I want some to
be ints or floats that get passed directly to the arguments of my
command-line script on the slave machine(s).

For concreteness, here is a test CSV I was working with:

coords,qvals,numphi,numshots,nummolec,photons,parallel,output_filename
gold_5nm.coor,gold_qvals.txt,3600,10,1,0.25,12,out0.ring
gold_5nm.coor,gold_qvals.txt,3600,10,2,0.5,12,out1.ring
gold_5nm.coor,gold_qvals.txt,3600,10,4,1.0,12,out2.ring
gold_5nm.coor,gold_qvals.txt,3600,10,8,2.0,12,out3.ring
gold_5nm.coor,gold_qvals.txt,3600,10,16,4.0,12,out4.ring

and my (non-functional) swift script, which will show what I was trying to
do:

# shoot.swift

type messagefile;
type pdbfile;
type shotfile;

type shootargs{
    pdbfile coords;
    messagefile qvals;
    int numphi;
    int numshots;
    int nummolec;
    int photons;
    int parallel;
    string output_filename;
}

app (shotfile outputfile) shootsim (shootargs args) {
     polshoot "-s" @args.coords "-f" @args.qvals "-x" args.numphi "-n"
args.numshots "-m" args.nummolec "-g" args.photons "-p" args.parallel "-o"
@outputfile;
}


shootargs myargs[] <csv_mapper;file="particles_per_shot.csv">;

foreach a in myargs {
    shotfile o; // this could be something like myargs.output_filename
    o = shootsim(a);
}

I'm wondering if someone who's worked a bit with swift can give me a
recommendation on how to proceed. Right now I'm playing with just writing a
huge number of flat text files, each one containing the parameter flags
that will then get cat'd into the arguments of my command-line script
"polshoot" on the slave end. This is inelegant for obvious reasons, since
I'll have a huge number of input files and no easy way to keep track of
which input matches what output...

If anyone has advice, I'm all ears!

Thanks,

TJ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20130610/5f78e087/attachment.html>