[Swift-devel] switchable copious provenance logging

[Ian Foster]

PS: This could be an interesting research project, perhaps?


On Apr 21, 2009, at 7:44 AM, Ian Foster wrote:

> Given the fact that Swift programs describe the structure of  
> computations, we should be able to compress logs considerably, by  
> reference to the program. E.g., the program:
>
> f() {
> 	g();
> 	h();
> }
>
> with no arguments will always (ignoring errors) execute g and h if f  
> is called. So we could just record that f() has been called.
>
> I can imagine taking ideas of that sort quite a long way.
>
> Now if f() has arguments, things get more complex. But one could  
> record subsets of argument information maybe?
>
> Ian.
>
>
> On Apr 21, 2009, at 7:28 AM, Michael Wilde wrote:
>
>> My perhaps uninformed and random observation on logging:
>>
>> the debug log seems very noisy, and it seems whats needed for  
>> provenance (ie, the major events, not all the minor ones) could be  
>> made much more compact.
>>
>> I think ultimately we want 2 control bits on logging:
>>
>> - debug level
>> - provenance (on/off) but maybe a level as well.
>>
>> Its possible these should go to separate channels, or be separable  
>> by a clear descriptor on the log line (for space management)
>>
>> It seems that while Swift is stll a young system, having debug on  
>> all the time is useful, so that error info gets captured without  
>> having to do re-runs. It seems that for most work we can afford the  
>> time overhead of these logs; the space overhead may become  
>> burdensome.
>>
>> Is the provenance logging making the size of the log unmanageable?
>>
>> On 4/21/09 3:06 AM, Ben Clifford wrote:
>>> As I work on provenance, the amount of log output for that becomes  
>>> larger.
>>> Two things are probably interesting wrt that:
>>>  i) make logging of provenance stuff switchable to on/off
>>> ii) move the provenance related log information to a separate file
>>> Comments.
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20090421/047553ad/attachment.html>