[petsc-users] Scalability

[Barry Smith]

   Yes the more the values are near the diagonal likely the better scaling. Also the number of nonzeros per row, the higher that number the better the scaling.

   Barry


> On Feb 13, 2026, at 8:43 AM, Matthew Knepley <knepley at gmail.com> wrote:
> 
> On Fri, Feb 13, 2026 at 2:14 AM SCOTTO Alexandre <alexandre.scotto at irt-saintexupery.com <mailto:alexandre.scotto at irt-saintexupery.com>> wrote:
>> Dear Matthew, Barry,
>> 
>>  
>> 
>> Thank you for your answers. The question of the problem size was part of my concern regarding the relevance of the quick test setup, I am going to increase the size in the suggested way to see the difference.
>> 
>>  
>> 
>> Regarding the sparsity pattern, I assume that the more “diagonal” the matrix is the better the speedup, is this is a correct rule of thumb?
>> 
> 
> What I was referring to was the density. The pattern has implications for the cache efficiency. Here is a good paper explaining what is going on:
> 
>   https://urldefense.us/v3/__https://d1wqtxts1xzle7.cloudfront.net/40652293/Toward_Realistic_Performance_Bounds_for_20151205-5192-8jxqcg-libre.pdf?1449311168=&response-content-disposition=inline*3B*filename*3DToward_realistic_performance_bounds_for.pdf&Expires=1770993790&Signature=ITtMQ-YNb5x*ZZnYof32wXbghpN9y5Bf50*ioozZi6O7GXATT4e4wApHuDX0qsrED1Pv--bv*rXFkMz9BpeGHP491X-qcDdKbRNxp7tg2zhKMwTeGpzzUCDV6UGjWcof39UCWzBSgNDhC35BVObFeDelhewIvn0dNI9O-Msr3wOjO51yDYzh1KJO-oTZ6mIDIYDL8S8ioLhnL0z6ec-3dQOmdDJfV6Vty3gkMJAjAhkhUNst2JEqIuRuygYGizCuVhYksH3p-51et7FWtu043MTmBO6lRCbKodbWMGBXvKe8Kox03NDQ2fs5-ClAWTwjd6VTiGpPq6PxP0a9UPvWZQ__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA__;JSslfn5-!!G_uCfscf7eWS!axZVqLk0h37e_aRGUtvMl_doja_7Vw4wdRhxvWhzyvFMozVXizBarj_RQS-_qxWkNTGjz50ihtujnoqWm896D_g$ 
> 
>   Thanks,
> 
>      Matt
>  
>> Best regards,
>> 
>> Alexandre.
>> 
>>  
>> 
>> De : Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>>
>> Envoyé : jeudi 12 février 2026 17:10
>> À : Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>>
>> Cc : SCOTTO Alexandre <alexandre.scotto at irt-saintexupery.com <mailto:alexandre.scotto at irt-saintexupery.com>>; petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>
>> Objet : Re: [petsc-users] Scalability
>> 
>>  
>> 
>>  
>> 
>>    The problem size is also very small. Typically one cannot get speedup when the number of variables per MPI rank is below on the order of 10,000. In your 64 process case you only have 390 variables. I would be stunned with any kind of speedup for such sizes. Run a problem at least 10 times bigger, better yet 20 times.
>> 
>>  
>> 
>> 
>> 
>> 
>> On Feb 12, 2026, at 9:00 AM, Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>> wrote:
>> 
>>  
>> 
>> On Thu, Feb 12, 2026 at 6:48 AM SCOTTO Alexandre via petsc-users <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>> wrote:
>> 
>> Dear PETSc community,
>> 
>>  
>> 
>> I have conducted a quick strong scalability-like test on direct and adjoint matrix-vector product with a 25,000 x 25,000 sparse matrix, distributed over 2, 4, …, 32 and 64 processes and the results I obtained were not so great.
>> 
>> I am not very confident in my setup, so a as a matter of reference, is there any available results on weak and strong scalability of PETSc.Mat mult() and multTranspose() operations?
>> 
>>  
>> 
>> 1. This behavior depends on available bandwidth, not on cores. Do you know the bandwidth for your configurations?
>> 
>>  
>> 
>> 2. Strong scaling depends heavily on matrix sparsity. If inevitably declines, but slower with more work to do.
>> 
>>  
>> 
>> 3. We published a paper on performance recently: https://urldefense.us/v3/__https://www.sciencedirect.com/science/article/abs/pii/S016781912100079X__;!!G_uCfscf7eWS!axZVqLk0h37e_aRGUtvMl_doja_7Vw4wdRhxvWhzyvFMozVXizBarj_RQS-_qxWkNTGjz50ihtujnoqWSwoa0nA$  <https://urldefense.us/v3/__https:/www.sciencedirect.com/science/article/abs/pii/S016781912100079X__;!!G_uCfscf7eWS!Zr5jUpk1srGDF2h9mXmw_GIn1OFZ2g3APzC0JHZREcxRzzy-2Oz2yyBWtzSI6F21kV4W_ubmc7A0NIIoVXnb$>
>>  
>> 
>>   Thanks,
>> 
>>  
>> 
>>       Matt 
>> 
>>  
>> 
>> Best regards,
>> 
>> Alexandre.
>> 
>> 
>> 
>>  
>> 
>> -- 
>> 
>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>> -- Norbert Wiener
>> 
>>  
>> 
>> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!axZVqLk0h37e_aRGUtvMl_doja_7Vw4wdRhxvWhzyvFMozVXizBarj_RQS-_qxWkNTGjz50ihtujnoqWiVlDisA$  <https://urldefense.us/v3/__http:/www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Zr5jUpk1srGDF2h9mXmw_GIn1OFZ2g3APzC0JHZREcxRzzy-2Oz2yyBWtzSI6F21kV4W_ubmc7A0NEclfLI_$>
>>  
>> 
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
> 
> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!axZVqLk0h37e_aRGUtvMl_doja_7Vw4wdRhxvWhzyvFMozVXizBarj_RQS-_qxWkNTGjz50ihtujnoqWiVlDisA$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!axZVqLk0h37e_aRGUtvMl_doja_7Vw4wdRhxvWhzyvFMozVXizBarj_RQS-_qxWkNTGjz50ihtujnoqWfV1bHoU$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20260213/20ee4a51/attachment-0001.html>