[Swift-devel] switchable copious provenance logging

[Ian Foster]

Given the fact that Swift programs describe the structure of  
computations, we should be able to compress logs considerably, by  
reference to the program. E.g., the program:

f() {
	g();
	h();
}

with no arguments will always (ignoring errors) execute g and h if f  
is called. So we could just record that f() has been called.

I can imagine taking ideas of that sort quite a long way.

Now if f() has arguments, things get more complex. But one could  
record subsets of argument information maybe?

Ian.


On Apr 21, 2009, at 7:28 AM, Michael Wilde wrote:

> My perhaps uninformed and random observation on logging:
>
> the debug log seems very noisy, and it seems whats needed for  
> provenance (ie, the major events, not all the minor ones) could be  
> made much more compact.
>
> I think ultimately we want 2 control bits on logging:
>
> - debug level
> - provenance (on/off) but maybe a level as well.
>
> Its possible these should go to separate channels, or be separable  
> by a clear descriptor on the log line (for space management)
>
> It seems that while Swift is stll a young system, having debug on  
> all the time is useful, so that error info gets captured without  
> having to do re-runs. It seems that for most work we can afford the  
> time overhead of these logs; the space overhead may become burdensome.
>
> Is the provenance logging making the size of the log unmanageable?
>
> On 4/21/09 3:06 AM, Ben Clifford wrote:
>> As I work on provenance, the amount of log output for that becomes  
>> larger.
>> Two things are probably interesting wrt that:
>>  i) make logging of provenance stuff switchable to on/off
>> ii) move the provenance related log information to a separate file
>> Comments.
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20090421/49295734/attachment.html>