[Swift-user] Swift for large-scale batch jobs?

Michael McCracken michael.mccracken at gmail.com
Wed Sep 5 17:21:16 CDT 2007


Hi, I'm studying the workflow of large-scale, 'capability'-class
experiments, specifically using workflow time prediction to support
planning and system choice for experiments that are too large for test
runs. I'm wondering if the kinds of workflows I'm studying could be
implemented in swift.

I'm a bit under-informed of the state of globus and related
technologies, so please excuse me if this should be obvious:
Can swift scripts control submission of jobs to a batch queue (local
or remote), either sequentially linked or independent?

The experiments I'm working with involve many sequentially-dependent
full-system runs (thousands of processors), with large enough data to
require transfer to archive or secondary storage between runs.
Has anyone tried something like this in swift?

What I'd like to do, if swift can support experiments like that, is
build a tool to read swift scripts ( or the appropriate intermediate
form ) and generate task descriptions for my current tool, which
simulates large scale experiments to predict total time to solution
(including queue wait and network transfer). Any advice on which point
in the swift tool chain to start at would be helpful. I've scanned the
code, but it is a lot to digest.

Thanks,
-mike

-- 
Michael McCracken
UCSD CSE PhD Candidate
research: http://www.cse.ucsd.edu/~mmccrack/
misc: http://michael-mccracken.net/wp/



More information about the Swift-user mailing list