[petsc-users] Scalability

[Matthew Knepley]

On Fri, Feb 13, 2026 at 2:14 AM SCOTTO Alexandre <
alexandre.scotto at irt-saintexupery.com> wrote:

> Dear Matthew, Barry,
>
>
>
> Thank you for your answers. The question of the problem size was part of
> my concern regarding the relevance of the quick test setup, I am going to
> increase the size in the suggested way to see the difference.
>
>
>
> Regarding the sparsity pattern, I assume that the more “diagonal” the
> matrix is the better the speedup, is this is a correct rule of thumb?
>

What I was referring to was the density. The pattern has implications for
the cache efficiency. Here is a good paper explaining what is going on:


https://urldefense.us/v3/__https://d1wqtxts1xzle7.cloudfront.net/40652293/Toward_Realistic_Performance_Bounds_for_20151205-5192-8jxqcg-libre.pdf?1449311168=&response-content-disposition=inline*3B*filename*3DToward_realistic_performance_bounds_for.pdf&Expires=1770993790&Signature=ITtMQ-YNb5x*ZZnYof32wXbghpN9y5Bf50*ioozZi6O7GXATT4e4wApHuDX0qsrED1Pv--bv*rXFkMz9BpeGHP491X-qcDdKbRNxp7tg2zhKMwTeGpzzUCDV6UGjWcof39UCWzBSgNDhC35BVObFeDelhewIvn0dNI9O-Msr3wOjO51yDYzh1KJO-oTZ6mIDIYDL8S8ioLhnL0z6ec-3dQOmdDJfV6Vty3gkMJAjAhkhUNst2JEqIuRuygYGizCuVhYksH3p-51et7FWtu043MTmBO6lRCbKodbWMGBXvKe8Kox03NDQ2fs5-ClAWTwjd6VTiGpPq6PxP0a9UPvWZQ__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA__;JSslfn5-!!G_uCfscf7eWS!Zyta7xbfKs3GEHTK5_8CF8c_Fz0cB2U8ek1mg8Vb0LsqwXKq9nGutXOd3rCZw5mI5nmdiPAXXPUWo1vSYcsq$ 

  Thanks,

     Matt


> Best regards,
>
> Alexandre.
>
>
>
> *De :* Barry Smith <bsmith at petsc.dev>
> *Envoyé :* jeudi 12 février 2026 17:10
> *À :* Matthew Knepley <knepley at gmail.com>
> *Cc :* SCOTTO Alexandre <alexandre.scotto at irt-saintexupery.com>;
> petsc-users at mcs.anl.gov
> *Objet :* Re: [petsc-users] Scalability
>
>
>
>
>
>    The problem size is also very small. Typically one cannot get speedup
> when the number of variables per MPI rank is below on the order of 10,000.
> In your 64 process case you only have 390 variables. I would be stunned
> with any kind of speedup for such sizes. Run a problem at least 10 times
> bigger, better yet 20 times.
>
>
>
>
>
> On Feb 12, 2026, at 9:00 AM, Matthew Knepley <knepley at gmail.com> wrote:
>
>
>
> On Thu, Feb 12, 2026 at 6:48 AM SCOTTO Alexandre via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
> Dear PETSc community,
>
>
>
> I have conducted a quick strong scalability-like test on direct and
> adjoint matrix-vector product with a 25,000 x 25,000 sparse matrix,
> distributed over 2, 4, …, 32 and 64 processes and the results I obtained
> were not so great.
>
> I am not very confident in my setup, so a as a matter of reference, is
> there any available results on weak and strong scalability of PETSc.Mat
> mult() and multTranspose() operations?
>
>
>
> 1. This behavior depends on available bandwidth, not on cores. Do you know
> the bandwidth for your configurations?
>
>
>
> 2. Strong scaling depends heavily on matrix sparsity. If inevitably
> declines, but slower with more work to do.
>
>
>
> 3. We published a paper on performance recently:
> https://urldefense.us/v3/__https://www.sciencedirect.com/science/article/abs/pii/S016781912100079X__;!!G_uCfscf7eWS!Zyta7xbfKs3GEHTK5_8CF8c_Fz0cB2U8ek1mg8Vb0LsqwXKq9nGutXOd3rCZw5mI5nmdiPAXXPUWo5KkdLTl$ 
> <https://urldefense.us/v3/__https:/www.sciencedirect.com/science/article/abs/pii/S016781912100079X__;!!G_uCfscf7eWS!Zr5jUpk1srGDF2h9mXmw_GIn1OFZ2g3APzC0JHZREcxRzzy-2Oz2yyBWtzSI6F21kV4W_ubmc7A0NIIoVXnb$>
>
>
>
>   Thanks,
>
>
>
>       Matt
>
>
>
> Best regards,
>
> Alexandre.
>
>
>
>
> --
>
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
>
>
> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Zyta7xbfKs3GEHTK5_8CF8c_Fz0cB2U8ek1mg8Vb0LsqwXKq9nGutXOd3rCZw5mI5nmdiPAXXPUWo_B6tlSa$ 
> <https://urldefense.us/v3/__http:/www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Zr5jUpk1srGDF2h9mXmw_GIn1OFZ2g3APzC0JHZREcxRzzy-2Oz2yyBWtzSI6F21kV4W_ubmc7A0NEclfLI_$>
>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Zyta7xbfKs3GEHTK5_8CF8c_Fz0cB2U8ek1mg8Vb0LsqwXKq9nGutXOd3rCZw5mI5nmdiPAXXPUWo_B6tlSa$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Zyta7xbfKs3GEHTK5_8CF8c_Fz0cB2U8ek1mg8Vb0LsqwXKq9nGutXOd3rCZw5mI5nmdiPAXXPUWo0Xlde1W$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20260213/e49bd926/attachment.html>