[petsc-users] Smaller assemble time with increasing processors

[Runfeng Jin]

Hello!

When I use PETSc build a sbaij matrix, I find a strange thing. When I
increase the number of processors, the assemble time become smaller. All
these are totally same matrix. The assemble time mainly arouse from message
passing, which because I use dynamic workload that it is random for which
elements are computed by which processor.
But from instinct, if use more processors, then more possible that the
processor computes elements storing in other processors. But from the
output of log_view, It seems when use more processors, the processors
compute more elements storing in its local(infer from that, with more
processors, less total amount of passed messages).

What could cause this happened? Thank you!


 Following is the output of log_view for 64\128\256 processors. Every row
is time profiler of VecAssemblyEnd.

------------------------------------------------------------------------------------------------------------------------
processors                Count                      Time (sec)
                          Flop
                 --- Global ---                               --- Stage
----                Total
                              Max    Ratio         Max
Ratio                 Max  Ratio      Mess        AvgLen         Reduct
           %T %F %M %L %R         %T %F %M %L %R       Mflop/s
64                            1     1.0            2.3775e+02      1.0
             0.00e+00 0.0      6.2e+03    2.3e+04     9.0e+00
   52  0      1    1    2             52   0    1      1     2             0
128                          1     1.0            6.9945e+01      1.0
             0.00e+00 0.0      2.5e+04    1.1e+04     9.0e+00
  30   0      1     1  2              30   0    1       1    2             0
256                          1     1.0           1.7445e+01        1.0
            0.00e+00 0.0      9.9e+04     5.2e+03    9.0e+00
10   0      1     1  2              10   0    1        1   2             0

Runfeng Jin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230630/f49384b9/attachment-0001.html>