[Swift-devel] Reducing swift log size

[Yadu Nand Babuji]

Hi Mihael,

Here's the package I'm running:
http://users.rcc.uchicago.edu/~yadunand/swift-0.95-modded.tar.gz
(It has Cpu.java modified, as well as as a debugging line in 
libexec/swift-int-staging.k)

I shutdown the run once the logs were past 4gb, here's the link to the 
log :
http://users.rcc.uchicago.edu/~yadunand/run013_with_mods_to_Cpujava.tar.gz

Thanks,
-Yadu

On 06/11/2014 07:03 PM, Mihael Hategan wrote:
>>>> Sorry, I should have mentioned this, but can you please gzip them? It's
>>>> a bit much otherwise.
>>>>
>>>> Mihael
>>>>
>>>> On Wed, 2014-06-11 at 17:57 -0500, Yadu Nand Babuji wrote:
>>>>> Hi Mihael,
>>>>>
>>>>> I've got the logs for you.
>>>>>
>>>>> This time, i've run the 6001 tasks with 20s delay added, and was all run
>>>>> with swift-0.95-RC6 (from our website) :
>>>>>
>>>>> Normal run                 ->
>>>>> http://users.rcc.uchicago.edu/~yadunand/run010_swift_normal/run010.log (
>>>>> 6.1gb )
>>>>> With minimal logging ->
>>>>> http://users.rcc.uchicago.edu/~yadunand/run011_minimal_logging/run011.log (
>>>>> 7.5 gb )
>>>>> With minimal logging ->
>>>>> http://users.rcc.uchicago.edu/~yadunand/run012_minimal_plus_log4j/run012.log
>>>>> ( 845.4 kb )
>>>>> & log4j properties modified
>>>>>
>>>>> The run with minimal logging ran for ~15mins while the first run took
>>>>> ~12mins. That *might* explain why the one
>>>>> with minimal logging is larger.
>>>>>
>>>>> Thanks,
>>>>> Yadu
>>>>>
>>>>> On 06/11/2014 02:54 PM, Mihael Hategan wrote:
>>>>>> On Wed, 2014-06-11 at 14:46 -0500, Yadu Nand Babuji wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm running a proxy app for a user with 6000 tasks each taking a few
>>>>>>> milliseconds, and the log sizes are unusually large. When the tasks are
>>>>>>> set to take 20s, the total log size reaches ~7Gb.
>>>>>> 7GB? Wow. I'd like to see that. Can you upload and post link?
>>>>>>
>>>>>>> I tried setting -minimal.logging and -reduced.logging but still see
>>>>>>> debug lines in the log like:
>>>>>>> 2014-06-11 19:15:06,961+0000 DEBUG Cpu worker started:
>>>>>>> block=0611-1507050-000000 host=midway461 id=10
>>>>>>> 2014-06-11 19:15:06,961+0000 DEBUG Cpu ready for work:
>>>>>>> block=0611-1507050-000000 id=10
>>>>>>>
>>>>>>> Do we need DEBUG lines such as the ones listed above ? Is it reasonable
>>>>>>> to have these set by default to WARN ?
>>>>>> It is until there's a problem and then people ask for the opposite. We
>>>>>> should evaluate whether this belongs in reduced logging or not. But does
>>>>>> that really account for most of the 7G?
>>>>>>
>>>>>>> Secondly, setting -minimal.logging did not turn off these DEBUG lines
>>>>>>> and I had to set the following log4j.properties from DEBUG to WARN to
>>>>>>> remove most of the offending lines:
>>>>>> That sounds like a bug.
>>>>>>
>>>>>>> log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Cpu=WARN
>>>>>>> log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Block=WARN
>>>>>>>
>>>>>>> With 6001 tasks, each taking 2 ms or so:
>>>>>>> Swift without any changes to logging ->  440879 lines and  51Mb
>>>>>>> Swift with -minimal.logging                ->    83350  lines and 9.5Mb
>>>>>>> Swift with -minimal.logging and         ->      7625  lines and 791Kb
>>>>>>> Cpu, Block log4j properties set
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Yadu
>>>>>>> _______________________________________________
>>>>>>> Swift-devel mailing list
>>>>>>> Swift-devel at ci.uchicago.edu
>>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>