[petsc-users] timer in PETSc

[Barry Smith]

  Nothing is clear to me. One way to debug is put in just one stage at a time (comment out all others) and make sure each one generates consistent numbers by itself. Then add a second stage. You have to debug this just like any other bug... 

  Barry

> On Apr 10, 2015, at 3:58 PM, Fande Kong <fande.kong at colorado.edu> wrote:
> 
> Thanks. I did add my several events. From the following cases, we possibly can find an explanation.
> 
> (1) 
> 
> Summary of Stages:   ----- Time ------  ----- Flops -----  --- Messages ---  -- Message Lengths --  -- Reductions --
>                         Avg     %Total     Avg     %Total   counts   %Total     Avg         %Total   counts   %Total 
>  0:      Main Stage: 1.2115e+03  91.0%  8.6211e+12   2.3%  5.937e+07   3.2%  7.314e+03       77.2%  2.344e+03  10.1% 
>  1:        MG Apply: 1.1994e+02   9.0%  3.6819e+14  97.7%  1.780e+09  96.8%  2.160e+03       22.8%  2.080e+04  89.9% 
>  
> (2)    Summary of Stages:   ----- Time ------  ----- Flops -----  --- Messages ---  -- Message Lengths --  -- Reductions --
>                         Avg     %Total     Avg     %Total   counts   %Total     Avg         %Total   counts   %Total 
>  0:      Main Stage: 9.9736e+02  47.6%  2.4648e+13   0.8%  5.813e+07   1.0%  1.013e+03       32.5%  3.151e+03   2.7% 
>  1:        MG Apply: 1.0959e+03  52.4%  2.9005e+15  99.2%  5.734e+09  99.0%  2.102e+03       67.5%  1.114e+05  97.2% 
> 
> 
> In the fist case, we do not have any warnings and the second case produces a warning. I am using MG as a preconditioner. When the MG is cheap like case 1, the Main Stage time is larger than the MG stage. We do not have any warnings. If the MG is expensive, the Main Stage time is smaller than MG. We have a warning now because SNESSolve time is larger than Main Stage. SNESSolve time includes outer solver and MG preconditioner.
> 
> Why we do not add MG time to the Main Stage. Should the Main Stage be the total time?
> 
> Fande, 
> 
> On Fri, Apr 10, 2015 at 2:30 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
>   Are you adding your own events or stages or just running with -log_summary?
> 
>   If you added your own events or stages this likely happened because you changed between stages within an event so its start time is logged in one stage but the end time correct gets put into a different stage.
> 
>   Barry
> 
> > On Apr 10, 2015, at 3:22 PM, Fande Kong <fande.kong at colorado.edu> wrote:
> >
> > Hi all,
> >
> > I am running an application with more than 4,000 cores. I get a warning in the -log_summary: Warning -- total time of even greater than time of entire stage -- something is wrong with the timer.
> >
> > I have no any clues for this warning. Any suggestions to remove it?
> >
> > Thanks,
> >
> > Fande,
> 
>