<div dir="ltr">And if I want to run my code without debugging mode on, I have to build PETSc from the beginning with the corresponding configure options, right?<div><br></div><div>That is some headache for me, as I am using PETSc on a supercomputer and it is built with the default option, where debug mode is on.</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sun, Jan 5, 2014 at 1:52 AM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
The debug version does extra reductions for error checking, you should never look at -log_summary with a debug build.<br>
<br>
Normally MatMult() for MPIAIJ matrices has nearest neighbor communication so neither MatMult() or MatMultTranspose() has global reductions but if some scatters involve all entries then VecScatter does use global reductions in those cases. You could run a slimmed down run on 2 process with one process in the debugger and put a break point in MPI_Allreduce() and MPI_Reduce() to see when it is being triggered inside the MatMultTranspose(). Use -start_in_debugger -debugger_nodes 0<br>
<span class="HOEnZb"><font color="#888888"><br>
<br>
Barry<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On Jan 4, 2014, at 1:32 PM, R. Oğuz Selvitopi <<a href="mailto:roguzsel@gmail.com">roguzsel@gmail.com</a>> wrote:<br>
<br>
> Hello,<br>
><br>
> I am trying to understand the output generated by PETSc with -log_summary option.<br>
><br>
> Using PetscLogStageRegister/PetscLogStagePush/PetscLogStagePop I want to find out if there exists unnecessary communication in my code.<br>
> My problem is with understanding the number of reductions performed.<br>
><br>
> I have a solver whose stages are logged, and in the summary stages output, I get<br>
><br>
> Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --<br>
> Avg %Total Avg %Total counts %Total Avg %Total counts %Total<br>
> 4: Solver: 6.5625e-04 4.3% 4.2000e+02 59.7% 1.600e+01 23.2% 3.478e+00 14.5% 8.000e+00 5.3%<br>
><br>
> where it seems I have 8 reduction operations performed. But in the details of the stage events, I get:<br>
><br>
> --- Event Stage 4: Solver<br>
><br>
> Event Count Time (sec) Flops --- Global --- --- Stage --- Total<br>
> Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s<br>
> ------------------------------------------------------------------------------------------------------------------------<br>
> MatMult 1 1.0 1.2207e-04 1.8 4.10e+01 4.1 8.0e+00 1.5e+01 0.0e+00 1 16 12 7 0 15 27 50 50 0 1<br>
> MatMultTranspose 1 1.0 1.2112e-04 1.0 4.60e+01 3.8 8.0e+00 1.5e+01 2.0e+00 1 18 12 7 1 18 30 50 50 25 1<br>
> VecDot 3 1.0 2.6989e-04 1.2 2.90e+01 2.6 0.0e+00 0.0e+00 3.0e+00 2 12 0 0 2 36 20 0 0 38 0<br>
> VecSet 2 1.0 8.1062e-06 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 0<br>
> VecAXPY 2 1.0 3.7909e-05 1.3 2.00e+01 2.0 0.0e+00 0.0e+00 0.0e+00 0 9 0 0 0 5 15 0 0 0 2<br>
> VecAYPX 1 1.0 5.0068e-06 1.2 1.00e+01 1.7 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 1 8 0 0 0 6<br>
> VecScatterBegin 2 1.0 7.2956e-05 2.4 0.00e+00 0.0 1.6e+01 1.5e+01 0.0e+00 0 0 23 14 0 6 0100100 0 0<br>
> VecScatterEnd 2 1.0 9.5129e-05 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 10 0 0 0 0 0<br>
><br>
> It seems there are only 5 reductions.<br>
><br>
> But when I detail my log stages, it shows up VecAXPY/VecAYPX operations require reductions as well (I have two VecAXPY and a single VecAYPX, so 5+3 = 8).<br>
> (Whose logs I have not included here).<br>
><br>
> Normally these two operations should not require any reductions at all, as opposed to VecDot.<br>
><br>
> Do VecAXPY/VecAYPX require reductions? Is it because PETSc is compiled with the debugging option so that it performs additional checks that perform reductions?<br>
><br>
> Which is the correct number of reductions in above statistics, 5 or 8?<br>
><br>
> Moreover, why does MatMult require no reduction whereas MatMultTranspose requires two of them?<br>
><br>
> Thanks in advance.<br>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">---------<br><br>Oguz.</div>
</div>