<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div><br></div> Take a look at the discussion in <a href="https://petsc.gitlab.io/-/petsc/-/jobs/5814862879/artifacts/public/html/manual/streams.html">https://petsc.gitlab.io/-/petsc/-/jobs/5814862879/artifacts/public/html/manual/streams.html</a> and I suggest you run the streams benchmark from the branch <span style="font-size: large; caret-color: rgb(51, 50, 56); color: rgb(51, 50, 56);">barry/2023-09-15/fix-log-pcmpi on your machine to get a baseline for what kind of speedup you can expect. </span><div><font color="#333238" size="4"><span style="caret-color: rgb(51, 50, 56);"><br></span></font></div><div><font color="#333238" size="4"><span style="caret-color: rgb(51, 50, 56);"> Then let us know your thoughts.</span></font></div><div><font color="#333238" size="4"><span style="caret-color: rgb(51, 50, 56);"><br></span></font></div><div><font color="#333238" size="4"><span style="caret-color: rgb(51, 50, 56);"> Barry</span></font></div><div><font color="#333238" size="4"><span style="caret-color: rgb(51, 50, 56);"><br></span></font><div><br id="lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On Jan 11, 2024, at 11:37 AM, Stefano Zampini <stefano.zampini@gmail.com> wrote:</div><br class="Apple-interchange-newline"><div><div dir="auto">You are creating the matrix on the wrong communicator if you want it parallel. You are using PETSc.COMM_SELF</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 11, 2024, 19:28 Steffen Wilksen | Universitaet Bremen <<a href="mailto:swilksen@itp.uni-bremen.de">swilksen@itp.uni-bremen.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>
<div style="font-family:Arial;font-size:14px"><p>Hi all,<br>
<br>
I'm trying to do repeated matrix-vector-multiplication of large sparse matrices in python using petsc4py. Even the most simple method of parallelization, dividing up the calculation to run on multiple processes indenpendtly, does not seem to give a singnificant speed up for large matrices. I constructed a minimal working example, which I run using<br>
<br>
mpiexec -n N python parallel_example.py,<br>
<br>
where N is the number of processes. Instead of taking approximately the same time irrespective of the number of processes used, the calculation is much slower when starting more MPI processes. This translates to little to no speed up when splitting up a fixed number of calculations over N processes. As an example, running with N=1 takes 9s, while running with N=4 takes 34s. When running with smaller matrices, the problem is not as severe (only slower by a factor of 1.5 when setting MATSIZE=1e+5 instead of MATSIZE=1e+6). I get the same problems when just starting the script four times manually without using MPI.<br>
I attached both the script and the log file for running the script with N=4. Any help would be greatly appreciated. Calculations are done on my laptop, arch linux version 6.6.8 and PETSc version 3.20.2.<br>
<br>
Kind Regards<br>
Steffen<br type="_moz"></p>
</div>
</blockquote></div>
</div></blockquote></div><br></div></div></body></html>