[Swift-devel] Re: GRAM and Swift discussion this week?

[Ioan Raicu]

See below:

Tim Freeman wrote:
> On Tue, 22 May 2007 14:34:07 -0500
> Ioan Raicu <iraicu at cs.uchicago.edu> wrote:
>
>   
>> the many logs that we generate are quite hard for people to
>>       follow, and keep track of what each one contains; we fixed this by
>>       developing a GUI that  can connect to the GT4 container remotely
>>       and display relevant information!
>>     
>
> Is that something that could be used for other GT services?
>
>   
I guess so...
Here is a screen shot:
http://people.cs.uchicago.edu/~iraicu/research/Falkon/Falkon_GUI.gif


Essentially, all it does is it uses Java swing to paint the GUI, which 
has a bunch of text fields that get populated from data from the results 
of web service calls which are being polled against the GT4 service in 
question (Falkon in our case).  Its nothing fancy, but I bet something 
like this could be made for the GT4 container in general that would give 
basic container and host statistics!
>   
>>           * the provisioner currently creates a new thread for every job
>>             (resource allocation) it sends to GRAM4
>>                 o depending on which allocation strategy is used, this
>>                   might/might not be a problem on TG nodes
>>                 o in theory, we don't want more than 100 or so GRAM4
>>                   jobs in parallel running, but  if we choose the policy
>>                   in which each job allocates a single machine, then we
>>                   can easily surpass 100 jobs in parallel... all the
>>                   other policies, would be able to allocate 1K+, even
>>                   10K+ machines with less than 100 jobs in parallel, so
>>                   it could work perfectly fine even with the current
>>                   implementation; in the long run, this might be able to
>>                   be changed to a pool of threads in the provisioner!
>>     
>
> Are these threads just waiting on notifications?  
>
>   
Right... I took the easy way out and just created 1 thread per GRAM job, 
but it doesn't have to be this way, as you pointed out below.
> If so: you should be able to reduce this to one thread by subscribing with the
> same notification consumer EPR for each GRAM job and demuxing the result (demux
> based on the producer EPR that is passed to the class implementing
> NotifyCallback).  That way the thread that creates the GRAM job can disappear
> once it is done with the create call. 
>
>   
This is on my list of things to do, but I just haven't gotten around to 
fixing this!  It hasn't really been an issue with my current tests and 
usage scenarios, but needs to be addressed for the general case as 
people will likely hit this problem if we have enough users using Falkon.

Thanks,
Ioan
> Tim 
>
>   

-- 
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
       http://dsl.cs.uchicago.edu/
============================================
============================================


-- 
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
       http://dsl.cs.uchicago.edu/
============================================
============================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20070523/d564e7cb/attachment.html>