[petsc-users] Reductions in various vector and matrix kernels

[R. Oğuz Selvitopi]

Hello,

I am trying to understand the output generated by PETSc with -log_summary
option.

Using PetscLogStageRegister/PetscLogStagePush/PetscLogStagePop I want to
find out if there exists unnecessary communication in my code.
My problem is with understanding the number of reductions performed.

I have a solver whose stages are logged, and in the summary stages output,
I get

Summary of Stages:   ----- Time ------  ----- Flops -----  --- Messages ---
 -- Message Lengths --  -- Reductions --
                                 Avg     %Total     Avg     %Total   counts
  %Total     Avg         %Total   counts   %Total
4:          Solver: 6.5625e-04   4.3%  4.2000e+02  59.7%  1.600e+01  23.2%
 3.478e+00       14.5%  8.000e+00   5.3%

where it seems I have 8 reduction operations performed. But in the details
of the stage events, I get:

--- Event Stage 4: Solver

Event                Count      Time (sec)     Flops
      --- Global ---  --- Stage ---   Total
                   Max Ratio  Max     Ratio   Max  Ratio  Mess   Avg len
Reduct  %T %f %M %L %R  %T %f %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
MatMult                      1 1.0 1.2207e-04 1.8 4.10e+01 4.1 8.0e+00
1.5e+01 0.0e+00  1 16 12  7  0  15 27 50 50  0     1
MatMultTranspose       1 1.0 1.2112e-04 1.0 4.60e+01 3.8 8.0e+00 1.5e+01
2.0e+00  1 18 12  7  1  18 30 50 50 25     1
VecDot                       3 1.0 2.6989e-04 1.2 2.90e+01 2.6 0.0e+00
0.0e+00 3.0e+00  2 12  0  0  2  36 20  0  0 38     0
VecSet                       2 1.0 8.1062e-06 1.5 0.00e+00 0.0 0.0e+00
0.0e+00 0.0e+00  0  0  0  0  0   1  0  0  0  0     0
VecAXPY                   2 1.0 3.7909e-05 1.3 2.00e+01 2.0 0.0e+00 0.0e+00
0.0e+00  0  9  0  0  0   5 15  0  0  0     2
VecAYPX                   1 1.0 5.0068e-06 1.2 1.00e+01 1.7 0.0e+00 0.0e+00
0.0e+00  0  5  0  0  0   1  8  0  0  0     6
VecScatterBegin         2 1.0 7.2956e-05 2.4 0.00e+00 0.0 1.6e+01 1.5e+01
0.0e+00  0  0 23 14  0   6  0100100  0     0
VecScatterEnd           2 1.0 9.5129e-05 2.2 0.00e+00 0.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0  10  0  0  0  0     0

It seems there are only 5 reductions.

But when I detail my log stages, it shows up VecAXPY/VecAYPX operations
require reductions as well (I have two VecAXPY and a single VecAYPX, so 5+3
= 8).
(Whose logs I have not included here).

Normally these two operations should not require any reductions at all, as
opposed to VecDot.

Do VecAXPY/VecAYPX require reductions? Is it because PETSc is compiled with
the debugging option so that it performs additional checks that perform
reductions?

Which is the correct number of reductions in above statistics, 5 or 8?

Moreover, why does MatMult require no reduction whereas MatMultTranspose
requires two of them?

Thanks in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140104/84a4e51d/attachment.html>