[Swift-user] A question about design choice

[Jonathan Margoliash]

Along with a couple typedefs and an app definition, yes. Is there a better
way to do what I'm doing? Thanks,

Jonathan

On Mon, Oct 8, 2012 at 3:31 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:

> The swift script you have for the parallel jobs, is it a simple foreach
> loop?
>
> Mihael
>
> On Mon, 2012-10-08 at 14:29 -0500, Jonathan Margoliash wrote:
> > Hello swift-user,
> >
> >
> > I had a question about the design of my program. My program involves a
> > fair bit of matrix manipulations, so instead of writing the control
> > structure of the program in swift, I've written most of the logic and
> > data management in matlab scripts. The only functionality of swift
> > that I use is to parallelize large for-loops, while the code that runs
> > my swift scripts, and the apps that swift distributes inside each
> > for-loop, are written in matlab. In essence, I've created a program in
> > matlab which calls a swift script many times in serial, using swift to
> > distributing many jobs in parallel.
> >
> >
> > The issue I've come across is that after a certain number of serial
> > calls to swift, I need to do a significant amount of data crunching
> > before I can resume making serial swift calls. This data crunching is
> > resource-intensive enough that it can't be done on a log in node, but
> > not nearly intensive enough that it needs to be broken down into
> > component parts and parallelized. My question is, what is the correct
> > way of handling this situation? Does it make sense to write a separate
> > swift script whose body only contains a single app call, so I can get
> > this minor job run on one of the compute nodes? If I do it that way,
> > will there likely be a significant delay while the scheduler processes
> > my request, or is it likely to go through quickly because I'm only
> > asking for one node for a short period of time?
> >
> >
> > Thanks for your comments,
> >
> >
> > Jonathan
> > _______________________________________________
> > Swift-user mailing list
> > Swift-user at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20121008/1baa8b7d/attachment.html>