[Swift-user] A question about design choice

[Jonathan Margoliash]

Hello swift-user,

I had a question about the design of my program. My program involves a fair
bit of matrix manipulations, so instead of writing the control structure of
the program in swift, I've written most of the logic and data management in
matlab scripts. The only functionality of swift that I use is to
parallelize large for-loops, while the code that runs my swift scripts, and
the apps that swift distributes inside each for-loop, are written in
matlab. In essence, I've created a program in matlab which calls a swift
script many times in serial, using swift to distributing many jobs in
parallel.

The issue I've come across is that after a certain number of serial calls
to swift, I need to do a significant amount of data crunching before I can
resume making serial swift calls. This data crunching is resource-intensive
enough that it can't be done on a log in node, but not nearly intensive
enough that it needs to be broken down into component parts and
parallelized. My question is, what is the correct way of handling this
situation? Does it make sense to write a separate swift script whose body
only contains a single app call, so I can get this minor job run on one of
the compute nodes? If I do it that way, will there likely be a significant
delay while the scheduler processes my request, or is it likely to go
through quickly because I'm only asking for one node for a short period of
time?

Thanks for your comments,

Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20121008/0c5b7262/attachment.html>