[petsc-users] strong-scaling vs weak-scaling

[Justin Chang]

Thanks all. So this issue was one of our ATPESC2015 exam questions, and
turned some friends into foes. Most eventually fell into the strong-scale
is harder camp, but some of these "friends" also believed PETSc is *not*
capable of handling dense matrices and is not portable. Just wanted to hear
some expert opinions on this :)

Anyway, in one of my applications, I am comparing the performance of some
VI solvers (i.e., with variable bounds) with that of just standard linear
solves (i.e., no variable bounds) for 3D advection-diffusion equations in
highly heterogeneous and anisotropic porous media. The parallel efficiency
in the strong-sense is roughly the same but the parallel efficiency in the
weak-sense is significantly worse for VI solvers. I suppose one inference
that can be made is that VI solvers take longer to solver as the problem
size grows. And yes solver iteration counts do grow so that has some to do
with it.

As for these "dynamic range" plots, I tried something like this across 1
and 8 MPI processes with the following problem sizes for a 3D anisotropic
diffusion problem with CG/BoomerAMG:

1,331
9,261
29,791
68,921
132,651
226,981
357,911
531,441
753,571
1,030,301

Using a single Intel Xeon E5-2670 compute node for this. Attached is the
plot, but instead of flat or incline lines, i get concave down curves. If
my problem size gets too big, the N/time rate decreases, whereas for very
small problems it increases. I am guessing bandwidth limitation have
something to do with the decrease in performance. In that HPGMG
presentation you attached the other day, it seems the rate should decrease
as problem size decreases. Perhaps this study should be done with more MPI
processes?


On Mon, Aug 22, 2016 at 4:14 PM, Karl Rupp <rupp at iue.tuwien.ac.at> wrote:

> Hi Justin,
>
>
> I have seen some people claim that strong-scaling is harder to achieve
>> than weak scaling
>> (e.g., https://www.sharcnet.ca/help/index.php/Measuring_Parallel_Sc
>> aling_Performance)
>> and generally speaking it makes sense - communication overhead increases
>> with concurrency.
>>
>> However, we know that most PETSc solvers/applications are not only
>> memory-bandwidth bound, but may not scale as well w.r.t. problem size as
>> other solvers (e.g., ILU(0) may beat out GAMG for small elliptic
>> problems but GAMG will eventually beat out ILU(0) for larger problems),
>> so wouldn't weak-scaling not only be the more interesting but more
>> difficult performance metric to achieve? Strong-scaling issues arise
>> mostly from communication overhead but weak-scaling issues may come from
>> that and also solver/algorithmic scalability w.r.t problem size (e.g.,
>> problem size N takes 10*T seconds to compute but problem size 2*N takes
>> 50*T seconds to compute).
>>
>> In other words, if one were to propose or design a new algorithm/solver
>> capable of handling large-scale problems, would it be equally if not
>> more important to show the weak-scaling potential? Because if you really
>> think about it, a "truly efficient" algorithm will be less likely to
>> scale in the strong sense but computation time will be close to linearly
>> proportional to problem size (hence better scaling in the weak-sense).
>> It seems if I am trying to convince someone that a proposed
>> computational framework is "high performing" without getting too deep
>> into performance modeling, a poor parallel efficiency (arising due to
>> good sequential efficiency) in the strong sense may not look promising.
>>
>
> These are all valid thoughts. Let me add another perspective: If you are
> only interested in the machine aspects of scaling, you could run for a
> fixed number of solver iterations. That allows you to focus on the actual
> computational work done and your results will exclusively reflect the
> machine's performance. Thus, even though fixing solver iterations and thus
> not running solvers to convergence is a bad shortcut from the solver point
> of view, it can be a handy way of eliminating algorithmic fluctuations.
> (Clearly, this simplistic approach has not only been used, but also
> abused...)
>
> Best regards,
> Karli
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160822/ec4d1d8a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: example_dynamic_range.eps
Type: application/postscript
Size: 87684 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160822/ec4d1d8a/attachment-0001.eps>