[Swift-devel] Re: GRAM and Swift discussion this week?

[Mihael Hategan]

On 05/23/2007 06:10:59 AM, Tim Freeman wrote:
> On Tue, 22 May 2007 14:34:07 -0500
> Ioan Raicu <iraicu at cs.uchicago.edu> wrote:
> 
> > the many logs that we generate are quite hard for people to
> >       follow, and keep track of what each one contains; we fixed
> this by
> >       developing a GUI that  can connect to the GT4 container
> remotely
> >       and display relevant information!
> 
> Is that something that could be used for other GT services?
> 
> 
> >           * the provisioner currently creates a new thread for every
> job
> >             (resource allocation) it sends to GRAM4
> >                 o depending on which allocation strategy is used,
> this
> >                   might/might not be a problem on TG nodes
> >                 o in theory, we don't want more than 100 or so GRAM4
> >                   jobs in parallel running, but  if we choose the
> policy
> >                   in which each job allocates a single machine, then
> we
> >                   can easily surpass 100 jobs in parallel... all the
> >                   other policies, would be able to allocate 1K+,
> even
> >                   10K+ machines with less than 100 jobs in parallel,
> so
> >                   it could work perfectly fine even with the current
> >                   implementation; in the long run, this might be
> able to
> >                   be changed to a pool of threads in the
> provisioner!
> 
> Are these threads just waiting on notifications?
> 
> If so: you should be able to reduce this to one thread by subscribing
> with the
> same notification consumer EPR for each GRAM job and demuxing the
> result (demux
> based on the producer EPR that is passed to the class implementing
> NotifyCallback).  That way the thread that creates the GRAM job can
> disappear
> once it is done with the create call.

I'd recommend the CoG abstractions. They do exactly that, but hide all  
the details.

Mihael

> 
> Tim
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 
>