[Swift-devel] Swift Performance Data

[Ioan Raicu]

Ben Clifford wrote:
> On Mon, 25 Jun 2007, Ioan Raicu wrote:
>
>   
>>  On the other hand, the logs
>> that are geared towards graphing them (2) are mostly based on fixed time
>> intervals, and a few are based on events. 
>>     
>
> right. I think there's a need for both (eg. compare task queue lengths or 
> CPU load against job lifetime lines).
>
>   
>> also relatively easy to add new state information to log to these existing
>> logs since they are all localized in a few places, with little effort, I can
>> add new metrics to monitor
>>     
>
> but only when those metrics are somehow associated with Falkon? One of the 
> interesting things to do, I think, is to be able to get a job lifetime 
> line that goes from when Swift decides the job exists all the way through 
> to when Swift decides the job has finished, with the two/three colour job 
> lines for jobs being inside Falkon as part of that lifetime line.
>   
Right, and I think we can do this from the Swift logs, including the 
preprocessing time in Swift, the postprocessing time, plus the 
end-to-end time the task spent in Falkon, etc...  the logs that I 
mentioned are Falkon specific, and the logs in Swift that generate this 
kind of information I believe are parsed from the debug/info logs (human 
readable) to come up with the machine readable logs for graphing.  We 
(Yong and I) had some trouble in the past generating these graphs from 
the Swift logs as the logs did not always contain all the information we 
needed to draw the graph, or the parsing would fail, and we had to 
manually fix the problem in the logs and try again the parsing!
>   
>> On the other hand, my debug logs (1-4) are all handled via log4j, look more
>> like the traditional logs that log4j generates and people are accustomed to,
>> but from my point of view, these are tedious and error-prone to parse for
>> graphing purposes.
>>     
>
> log4j can easily be configured to output different formats - so we could 
> have human readable logs in one format and machine readable logs logging 
> different information in a different format, I think.
>   
OK, thats good!
>   
>> Does this distinction (human readable vs. machine readable) between logs exist
>> in Swift? 
>>     
>
> A little bit. The data in the swift/karajan logs is mostly intended for 
> human consumption; the data in kickstart records is very much more 
> structured and intended to be both human readable and machine readable.
>
> More machine readable edge and level based logging from inside swift and 
> inside karajan could be useful, I think.
>   
Right, but kickstart logs are all in separate files, so to really make 
sense of them in a programatic way and to plot them on 1 graph, there 
needs to be an aggreting step that either just merges these files 
together in some orderly way, or it mights even usmmarize the data for 
easier graphing.  From my understanding of the kickstart records, I 
think its hard to generate overview graphs of an entire run due to the 
fact that they are kept in many files.

Ioan

-- 
============================================
Ioan Raicu
Ph.D. Student
============================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
============================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
       http://dsl.cs.uchicago.edu/
============================================
============================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20070625/98c75930/attachment.html>