[Swift-devel] Reducing swift log size

[Mihael Hategan]

Ah!

Well, so the trick with two coaster services on localhost doesn't really
work well unless you use "ssh:", and this is a good example why. 

In your case you can avoid it easily if you change your first pool to
use the local provider instead of the coaster provider, since you don't
really need coasters to run locally.

Mihael


On Thu, 2014-06-12 at 12:11 -0500, Yadu Nand Babuji wrote:
> Hi Mihael,
> 
> Here's the package I'm running:
> http://users.rcc.uchicago.edu/~yadunand/swift-0.95-modded.tar.gz
> (It has Cpu.java modified, as well as as a debugging line in 
> libexec/swift-int-staging.k)
> 
> I shutdown the run once the logs were past 4gb, here's the link to the 
> log :
> http://users.rcc.uchicago.edu/~yadunand/run013_with_mods_to_Cpujava.tar.gz
> 
> Thanks,
> -Yadu
> 
> On 06/11/2014 07:03 PM, Mihael Hategan wrote:
> >>>> Sorry, I should have mentioned this, but can you please gzip them? It's
> >>>> a bit much otherwise.
> >>>>
> >>>> Mihael
> >>>>
> >>>> On Wed, 2014-06-11 at 17:57 -0500, Yadu Nand Babuji wrote:
> >>>>> Hi Mihael,
> >>>>>
> >>>>> I've got the logs for you.
> >>>>>
> >>>>> This time, i've run the 6001 tasks with 20s delay added, and was all run
> >>>>> with swift-0.95-RC6 (from our website) :
> >>>>>
> >>>>> Normal run                 ->
> >>>>> http://users.rcc.uchicago.edu/~yadunand/run010_swift_normal/run010.log (
> >>>>> 6.1gb )
> >>>>> With minimal logging ->
> >>>>> http://users.rcc.uchicago.edu/~yadunand/run011_minimal_logging/run011.log (
> >>>>> 7.5 gb )
> >>>>> With minimal logging ->
> >>>>> http://users.rcc.uchicago.edu/~yadunand/run012_minimal_plus_log4j/run012.log
> >>>>> ( 845.4 kb )
> >>>>> & log4j properties modified
> >>>>>
> >>>>> The run with minimal logging ran for ~15mins while the first run took
> >>>>> ~12mins. That *might* explain why the one
> >>>>> with minimal logging is larger.
> >>>>>
> >>>>> Thanks,
> >>>>> Yadu
> >>>>>
> >>>>> On 06/11/2014 02:54 PM, Mihael Hategan wrote:
> >>>>>> On Wed, 2014-06-11 at 14:46 -0500, Yadu Nand Babuji wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I'm running a proxy app for a user with 6000 tasks each taking a few
> >>>>>>> milliseconds, and the log sizes are unusually large. When the tasks are
> >>>>>>> set to take 20s, the total log size reaches ~7Gb.
> >>>>>> 7GB? Wow. I'd like to see that. Can you upload and post link?
> >>>>>>
> >>>>>>> I tried setting -minimal.logging and -reduced.logging but still see
> >>>>>>> debug lines in the log like:
> >>>>>>> 2014-06-11 19:15:06,961+0000 DEBUG Cpu worker started:
> >>>>>>> block=0611-1507050-000000 host=midway461 id=10
> >>>>>>> 2014-06-11 19:15:06,961+0000 DEBUG Cpu ready for work:
> >>>>>>> block=0611-1507050-000000 id=10
> >>>>>>>
> >>>>>>> Do we need DEBUG lines such as the ones listed above ? Is it reasonable
> >>>>>>> to have these set by default to WARN ?
> >>>>>> It is until there's a problem and then people ask for the opposite. We
> >>>>>> should evaluate whether this belongs in reduced logging or not. But does
> >>>>>> that really account for most of the 7G?
> >>>>>>
> >>>>>>> Secondly, setting -minimal.logging did not turn off these DEBUG lines
> >>>>>>> and I had to set the following log4j.properties from DEBUG to WARN to
> >>>>>>> remove most of the offending lines:
> >>>>>> That sounds like a bug.
> >>>>>>
> >>>>>>> log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Cpu=WARN
> >>>>>>> log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Block=WARN
> >>>>>>>
> >>>>>>> With 6001 tasks, each taking 2 ms or so:
> >>>>>>> Swift without any changes to logging ->  440879 lines and  51Mb
> >>>>>>> Swift with -minimal.logging                ->    83350  lines and 9.5Mb
> >>>>>>> Swift with -minimal.logging and         ->      7625  lines and 791Kb
> >>>>>>> Cpu, Block log4j properties set
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Yadu
> >>>>>>> _______________________________________________
> >>>>>>> Swift-devel mailing list
> >>>>>>> Swift-devel at ci.uchicago.edu
> >>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >>
> >> _______________________________________________
> >> Swift-devel mailing list
> >> Swift-devel at ci.uchicago.edu
> >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >
>