<div dir="ltr">Hello Matt, <div>thanks for your quick reply. My situation is a bit different. What I am solving is a generalized eigenvalue problem. I only recently switched from scipy.eigs to slepc, because slepc support parallel eigenvalue problem. In other word, I am adapting my old code to utilizing slepc/petsc parallel support. To have a first test, I saved the matrix from eigenvalue problem with old code. Then read in this matrix for slepc/petsc validation. That's why I need to scatter it to processors. Do you have any idea how can I achieve this? </div><div>with regards</div><div>Yongxiang</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 6 April 2018 at 15:00, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Fri, Apr 6, 2018 at 8:44 AM, Yongxiang Wu <span dir="ltr"><<a href="mailto:yongxiang27@gmail.com" target="_blank">yongxiang27@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><p style="margin:0px;padding:0px;word-wrap:break-word;color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:-0.07px;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">Hello, everyone,</p><p style="margin:0px;padding:0px;word-wrap:break-word;color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:-0.07px;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">I already have a scipy sparse square matrix<span> </span><code style="font-family:SFMono-Medium,"SF Mono","Segoe UI Mono","Roboto Mono","Ubuntu Mono",Menlo,Courier,monospace;font-size:12px;line-height:1.4;padding:1px 3px;border:0px;border-radius:3px;background:rgb(244,245,247);box-sizing:border-box;display:inline-block;max-width:100%;overflow-x:auto;vertical-align:bottom;white-space:nowrap">L0</code><span> </span>. Since my problem is large, parallel run is preferred. My Question is, how can I scatter my<span> </span><code style="font-family:SFMono-Medium,"SF Mono","Segoe UI Mono","Roboto Mono","Ubuntu Mono",Menlo,Courier,monospace;font-size:12px;line-height:1.4;padding:1px 3px;border:0px;border-radius:3px;background:rgb(244,245,247);box-sizing:border-box;display:inline-block;max-width:100%;overflow-x:auto;vertical-align:bottom;white-space:nowrap">L0</code><span> </span>to each of the processors? In the following code, I can get the indices of the localized part of the matrix. In the tutorial, the matrix element are directly assign with value, but in my case, the matrix are so large, assign each element in loop (commented code) is not efficient. So if any function would do the mpi scatter work? </p></div></blockquote><div><br></div></span><div>Hi Yongxiang,</div><div><br></div><div>It would be really anomalous for what you propose to results in any speedup at all. If the matrix is large,</div><div>it will not fit on one process. Any speedup from using more processes will be eaten up by the time to</div><div>communicate the matrix. I would focus on generating the matrix in parallel.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><p style="margin:0px;padding:0px;word-wrap:break-word;color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:-0.07px;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">With regards and <span style="letter-spacing:-0.07px">Thanks.</span></p><div class="m_-3129048948513139848m_-6351947006644004745gmail-codehilite m_-3129048948513139848m_-6351947006644004745gmail-language-python" style="margin:10px 0px 0px;padding:0px;color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:-0.07px;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><pre style="margin:0px;padding:5px 10px;font-family:SFMono-Medium,"SF Mono","Segoe UI Mono","Roboto Mono","Ubuntu Mono",Menlo,Courier,monospace;font-size:12px;line-height:1.4;background:rgb(244,245,247);border:0px;border-radius:3px;overflow-x:auto;word-wrap:normal;letter-spacing:0px"><span></span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-kn" style="font-weight:700">import</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-nn" style="color:rgb(85,85,85)">sys</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">,</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-nn" style="color:rgb(85,85,85)">slepc4py</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-n">slepc4py</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">init</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">(</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">sys</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">argv</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">)</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-kn" style="font-weight:700">from</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-nn" style="color:rgb(85,85,85)">petsc4py</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-kn" style="font-weight:700">import</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">PETSc</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-kn" style="font-weight:700">from</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-nn" style="color:rgb(85,85,85)">slepc4py</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-kn" style="font-weight:700">import</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">SLEPc</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-n">opts</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">=</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">PETSc</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">Options</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">()</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-n">opts</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">setValue</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">(</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-s1" style="color:rgb(187,136,68)">'-st_pc_factor_m<wbr>at_solver_package'</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">,</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-s1" style="color:rgb(187,136,68)">'mumps'</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">)</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-n">A</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">=</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">PETSc</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">Mat</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">()</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">createAIJ</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">(</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">size</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">=</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">L0</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n"><wbr>shape</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">,</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">comm</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">=</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">PETSc</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">COMM_WORLD</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">)</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-n">A</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">setUp</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">()</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-n">Istart</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">,</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">Iend</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">=</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">A</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">getOwnershipRange</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">()</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-c1" style="font-style:italic;color:rgb(153,153,136)"># for I in range(Istart,Iend):</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-c1" style="font-style:italic;color:rgb(153,153,136)"># for J in range(0,L0.shape[0]):</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-c1" style="font-style:italic;color:rgb(153,153,136)"># A[I,J] = L0[I,J]</span>
</pre></div><p style="margin:12px 0px 0px;padding:0px;word-wrap:break-word;color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:-0.07px;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">The flowing code, would make the assignment from the scipy sparse matrix<span> </span><code style="font-family:SFMono-Medium,"SF Mono","Segoe UI Mono","Roboto Mono","Ubuntu Mono",Menlo,Courier,monospace;font-size:12px;line-height:1.4;padding:1px 3px;border:0px;border-radius:3px;background:rgb(244,245,247);box-sizing:border-box;display:inline-block;max-width:100%;overflow-x:auto;vertical-align:bottom;white-space:nowrap">L0</code><span> </span>to PETSc matrix A. But this would only work for one process.</p><div class="m_-3129048948513139848m_-6351947006644004745gmail-codehilite m_-3129048948513139848m_-6351947006644004745gmail-language-python" style="margin:10px 0px 0px;padding:0px;color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:-0.07px;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><pre style="margin:0px;padding:5px 10px;font-family:SFMono-Medium,"SF Mono","Segoe UI Mono","Roboto Mono","Ubuntu Mono",Menlo,Courier,monospace;font-size:12px;line-height:1.4;background:rgb(244,245,247);border:0px;border-radius:3px;overflow-x:auto;word-wrap:normal;letter-spacing:0px"><span></span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">A</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">=</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">PETSc</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">Mat</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">()</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">createAIJ</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">(</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">size</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">=</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">L0</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n"><wbr>shape</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">,</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-n">csr</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">=</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">(</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">L0</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">indptr</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">,</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">L0</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">indices</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">,</span>
<span class="m_-3129048948513139848m_-6351947006644004745gmail-n">L0</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">data</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">),</span> <span class="m_-3129048948513139848m_-6351947006644004745gmail-n">comm</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">=</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">PETSc</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-o" style="font-weight:700">.</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-n">COMM_WORLD</span><span class="m_-3129048948513139848m_-6351947006644004745gmail-p">)</span></pre></div><br></div>
</blockquote></span></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div class="m_-3129048948513139848gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.caam.rice.edu/~mk51/" target="_blank">https://www.cse.buffalo.edu/~<wbr>knepley/</a><br></div></div></div></div></div>
</font></span></div></div>
</blockquote></div><br></div>