[Swift-devel] Re: Need precise throttle on local provider

[Mihael Hategan]

On Fri, 2010-10-01 at 09:38 -0700, Mihael Hategan wrote:
> > 
> > I disagree. This "slot" functionality is the primary goal. The
> > throttling approach is a poor workaround, as it forces me to create
> > one local pool per core, when what I really want is just one pool.
> 
> I am unsure what one thing has to do with the other. If you say
> poolCpus=4 and jobsPerCpu=1, you presumably get what you want. The issue
> isn't how you express the throttling, but where in the code the
> throttling is implemented. And I don't see what the mechanism used for
> submission (e.g. gt2, local, pbs - i.e. provider) has to do with how to
> distribute jobs to sites (the scheduler).

I now see better what you are saying.

I believe that there are two problems:
1. Enforcing a limited number of concurrent jobs on a site
2. Labeling the jobs in such a way that any two concurrent jobs always
have a distinct label.

I believe that this is a scenario that is very unlikely to occur in
normal swift usage. I would therefore be somewhat against writing
"generic" code to address (2) if a possibility exists to solve the
problem with the unmodified swift. Assuming (1), that possibility is
"virtual sites" which is what you are trying to do.

I don't feel strongly about this though. What do others think?

Mihael