[Swift-devel] CPU usage with provider-deef

[Mihael Hategan]

On Sat, 2007-06-16 at 12:03 -0500, Mike Wilde wrote:
> This should be fun, and a nice break from the I2U2 work that you've 
> been immersed in, Mihael.

I've already looked at the Falkon code and it's... a lot of code that
does stuff that I understand only in principle. What you want isn't
easy, and I have my reservations towards the amount of fun it involves.

That being said, Ioan, would it be possible to have a cleaned up version
of the code where there are no duplicate classes? It's hard for me to
figure what's relevant or not in that case. And perhaps dead
code/comments removed?

Mihael

> 
> Want to do a read-through soon, and send out comments for discussion 
> that can turn into a list of code improvements to bugzilize?
> 
> What I think is important about Falkon is that its working, its 
> proving out the value of the provisioned direct-scheduling approach 
> with numbers, and that its working for Ioan as a vehicle for his 
> research.
> 
> What we want to get from the effort is a) Ioan progresses towards 
> his PhD; b) the immediate needs of our app-users get met; and c) we 
> learn whats needed in architecture, protocol and algorithm for a 
> successful long-term approach to running swift programs efficiently.
> 
> Point is that everyone is open to changes and towards an eventual 
> re-design and re-write. This, Mihael, would be where you can 
> propose, design and implement the ideas you've expressed about 
> implementing provisioned direct-scheduling using Karajan's remote 
> execution mechanisms.
> 
> - Mike
> 
> 
> 
> 
> Ian Foster wrote, On 6/16/2007 10:59 AM:
> > It seems important that Ioan sit down with Mihael and work through the 
> > Falkon code to see where it can be simplified, improved, etc. I am sure 
> > that this will result in problems being identified and fixed that will 
> > otherwise cost us time later.
> > 
> > Mihael Hategan wrote:
> >> Yourkit (www.yourkit.com) has free licenses for open source projects for
> >> their profiler. Point them to a globus web page that has your name, and
> >> they'll send you the license. Alternatively, there are other profilers
> >> out there, and I strongly recommend using them on such issues.
> >>
> >> Mihael
> >>
> >> On Sat, 2007-06-16 at 10:00 -0500, Ioan Raicu wrote:
> >>   
> >>> Nope, I think this is a different problem, or at least a subset of the
> >>> problems we were having before.
> >>>
> >>> Since we fixed the CPU utilization, and we moved to a bigger box (4
> >>> CPUs with 2GB of memory), everything is happening in a timely fashion
> >>> (a few ms per notification delivery throughout the experiment).  Plus,
> >>> I believe the view is consistent (the same tasks look complete on both
> >>> ends) between Falkon and Swift, but we are still checking on this as
> >>> the run was made just last night for the 100 mol run.  We'll keep you
> >>> posted with what we find.
> >>>
> >>> Ioan
> >>>
> >>> Ben Clifford wrote: 
> >>>     
> >>>> On Sat, 16 Jun 2007, Ioan Raicu wrote:
> >>>>
> >>>>   
> >>>>       
> >>>>> having problems with the 100 molecule run in MolDyn.  Its not clear where the
> >>>>> problem is, on the surface Falkon looks fine... we are looking into where
> >>>>> everything breaks to cause Swift to not continue with the workflow to
> >>>>> completion!
> >>>>>     
> >>>>>         
> >>>> The same problem that you showed me the other day or different?
> >>>>
> >>>> with 'the same problem' being that falkon thinks all the jobs are done; 
> >>>> but that falkon's measure response time for sending completion 
> >>>> notifications gets approximately linearly longer over time and the swift 
> >>>> JVM uses ~100% and doesn't inidicate job completion at all after a certain 
> >>>> period.
> >>>>
> >>>> or different symptoms now?
> >>>>
> >>>>   
> >>>>       
> >>> -- 
> >>> ============================================
> >>> Ioan Raicu
> >>> Ph.D. Student
> >>> ============================================
> >>> Distributed Systems Laboratory
> >>> Computer Science Department
> >>> University of Chicago
> >>> 1100 E. 58th Street, Ryerson Hall
> >>> Chicago, IL 60637
> >>> ============================================
> >>> Email: iraicu at cs.uchicago.edu
> >>> Web:   http://www.cs.uchicago.edu/~iraicu
> >>>        http://dsl.cs.uchicago.edu/
> >>> ============================================
> >>> ============================================
> >>>     
> >>
> >>   
> > 
> > -- 
> > 
> >    Ian Foster, Director, Computation Institute
> > Argonne National Laboratory & University of Chicago
> > Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439
> > Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637
> > Tel: +1 630 252 4619.  Web: www.ci.uchicago.edu.
> >       Globus Alliance: www.globus.org.
> > 
> > 
> > ------------------------------------------------------------------------
> > 
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>