<div dir="ltr"><div dir="ltr">On Thu, Jul 10, 2025 at 11:29 AM Arun Soman Pillai - STFC UKRI via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:</div><div class="gmail_quote gmail_quote_container"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-8904305937024536206">
<div lang="EN-GB" style="overflow-wrap: break-word;">
<div class="m_8481806855935692819WordSection1">
<p class="MsoNormal" style="margin-bottom:12pt">Hi,<br>
I am trying to view a matrix object when running a job in parallel. The application is BOUT++ (<a href="https://urldefense.us/v3/__https://boutproject.github.io/__;!!G_uCfscf7eWS!Z-f_HrmUaFAwAK4e2pBspL2oRFtfq9O3GtRCoS_Nb1e9y5j2Vjq6THJTnoot7XiMeRAxlNhskW0QrMV7x_v_VsBXVAsGHRrIifxc$" target="_blank">https://boutproject.github.io/</a>) which uses petsc as solver.
<br>
While the option -mat_view gives a jumbled output on STDOUT from across the processors, </p></div></div></div></blockquote><div>This means that something is terribly wrong. I suspect that you are not actually running in parallel, but rather launching independent jobs. That can happen when mpiexec does not match the MPI library that you linked with.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-8904305937024536206"><div lang="EN-GB" style="overflow-wrap: break-word;"><div class="m_8481806855935692819WordSection1"><p class="MsoNormal" style="margin-bottom:12pt">redirecting output to a specific file results in race condition. The only option that works for me is
<i>-mat_view binary</i> with <i>-viewer_binary_filename <name></i><b>. </b>I am processing this binary file with petsc4py as follows:<b><u></u><u></u></b></p>
<p class="m_8481806855935692819p1"><span class="m_8481806855935692819s1">from</span><span class="m_8481806855935692819s2"> </span><span class="m_8481806855935692819s3">petsc</span><span class="m_8481806855935692819s2">4py
</span><span class="m_8481806855935692819s1">import</span><span class="m_8481806855935692819s2"> PETSc</span></p>
<p class="m_8481806855935692819p1"><span class="m_8481806855935692819s1">import</span><span class="m_8481806855935692819s2"> numpy </span><span class="m_8481806855935692819s4">as</span><span class="m_8481806855935692819s2"> np</span></p>
<p class="m_8481806855935692819p1"><span class="m_8481806855935692819s1">import</span><span class="m_8481806855935692819s2"> matplotlib.pyplot </span>
<span class="m_8481806855935692819s4">as</span><span class="m_8481806855935692819s2"> plt</span></p>
<p class="m_8481806855935692819p2"><span class="m_8481806855935692819s2">import</span><span class="m_8481806855935692819s5"> glob<u></u><u></u></span></p>
<p class="m_8481806855935692819p2"><span class="m_8481806855935692819s5"><u></u> <u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(186,96,24)">def</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(39,167,181)">read_binary</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">(filename):</span><span style="font-size:13.5pt;font-family:Menlo"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> viewer = PETSc.Viewer().createMPIIO(filename,
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)">"r"</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">, comm=PETSc.COMM_WORLD)<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> A = PETSc.Mat().load(viewer)<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> A.assemble()<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(39,167,181)">print</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">(</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)">"Matrix
type:"</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">, A.getType())<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(39,167,181)">print</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">(</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)">"Matrix
size:"</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">, A.getSize())<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> info = A.getInfo()<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(39,167,181)">print</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">(</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)">"Number
of nonzeros:"</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">, info[</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)">'nz_used'</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">])</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(39,167,181)">print</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">(</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)">"Nonzeros
allocated:"</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">, info[</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)">'nz_allocated'</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">])</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)"><u></u><u></u></span></p>
<p class="m_8481806855935692819p1"> m, n = A.getSize()<u></u><u></u></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> dense_mat = np.zeros((m, n), dtype=np.float64)<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> </span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(186,96,24)">
for</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> i
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(186,96,24)">in</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(39,167,181)">range</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">(</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(175,32,23)">m</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">):</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(186,96,24)"><u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> cols, vals = A.getRow(i)<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> dense_mat[i, cols] = vals<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"><u></u> <u></u></span></p>
<p class="m_8481806855935692819p1"> rstart, rend = A.getOwnershipRange()<u></u><u></u></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> rows, cols = [], []<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(186,96,24)">for</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> i
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(186,96,24)">in</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(39,167,181)">range</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">(rstart, rend):<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> cols_i, _ = A.getRow(i)<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> rows.extend([i] *
</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(39,167,181)">len</span><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">(cols_i))<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> cols.extend(cols_i)<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"><u></u> <u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> rows = np.array(rows)<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> cols = np.array(cols)<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"><u></u> <u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)"> return dense_mat, rows, cols<u></u><u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(186,96,24)"><u></u> <u></u></span></p>
<p class="MsoNormal" style="background:rgb(252,244,220)"><span style="font-size:13.5pt;font-family:Menlo;color:rgb(83,104,112)">A, rows, cols = read_binary(“matrix”)<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">This yields accurate number of non-zeros and matrix sparsity. However, the dense matrix turns out to be a null matrix. I wonder what is going wrong here.</p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Thank you, best regards</p>
<p class="MsoNormal">Arun Pillai</p>
</div>
</div>
</div></blockquote></div><div><br clear="all"></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!ZLeEFXZF4ASJ2hjoTAuK-DQjJkNz_wPtN8mH3P557R1p2cltUNodOb0wrXsWAcQlamre0C1SJy4cn5XgDHit$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>