<div dir="ltr"><div dir="ltr">On Tue, Jan 27, 2026 at 11:40 PM neil liu <<a href="mailto:liufield@gmail.com">liufield@gmail.com</a>> wrote:</div><div class="gmail_quote gmail_quote_container"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><font size="4">Thanks a lot, Matt. </font><div><p><font size="4">I assemble a matrix using <code>MatSetValuesLocal()</code>, where the cell-to-local mapping comes from <code>PetscSectionGetOffset()</code>.</font></p><p><font size="4">With 2 MPI ranks, Valgrind (rank 0) reports uninitialized data being passed to <code>MPI_Isend</code> during matrix assembly (inside <code>matstash.c</code>).</font></p><p><font size="4">Two observations:</font></p><ul><li><p><font size="4">If I hard-code the local mapping to always be <code>1</code>, the Valgrind warning disappears.</font></p></li><li><p><font size="4">The warning occurs with PETSc <strong>3.23.3</strong>, but disappears when switching to <strong>3.24.3</strong> under the same setup.</font></p></li></ul><p><font size="4"></font></p></div></div></blockquote><div>MPICH plays games with memory for efficiency reasons. We often see Valgrind warnings. Could you check if the default MPICH version changed between 3.23.3 and 3.24.3. That would be my first guess.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><p><font size="4">This suggests either uninitialized local indices/values on my side, or a change in PETSc between these versions affecting stash packing or index handling.</font></p><p><font size="4">Does this type of Valgrind warning usually point to user-provided local indices being uninitialized, and were there relevant changes in this area between 3.23.3 and 3.24.x?</font></p></div><div><span style="font-size:large"><br></span></div><div><span style="font-size:large">Thanks, </span></div><div><span style="font-size:large">Xiaodong </span></div><div><font size="4"><br></font></div><div>==683913== Syscall param write(buf) points to uninitialised byte(s)<br>==683913== at 0xCAD15A8: write (in /usr/lib64/<a href="https://urldefense.us/v3/__http://libc-2.28.so__;!!G_uCfscf7eWS!axRnfOFmHcGL3iKKS0tSlsif0QKDXBdDwXWrPhWj9HPh6Td-Xx10T5eHPX4YORD_ITemssQl-Kasmx2CkR8F$" target="_blank">libc-2.28.so</a>)<br>==683913== by 0xBB9B0A9: MPIDI_CH3I_Sock_write (sock.c:2622)<br>==683913== by 0xBBA12DF: MPIDI_CH3_iStartMsg (ch3_istartmsg.c:68)<br>==683913== by 0xBB55F82: MPIDI_CH3_RndvSend (ch3u_rndv.c:48)<br>==683913== by 0xBB70905: MPID_Isend (mpid_isend.c:159)<br>==683913== by 0xB83EDB7: internal_Isend_c (isend.c:273)<br>==683913== by 0xB83EFE9: PMPI_Isend_c (isend.c:363)<br>==683913== by 0x82CAC1A: MatStashBTSSend_Private (matstash.c:793)<br>==683913== by 0x6AA2BF8: PetscCommBuildTwoSidedFReq_Reference (mpits.c:311)<br>==683913== by 0x6AA9DAD: PetscCommBuildTwoSidedFReq (mpits.c:515)<br>==683913== by 0x82CEB46: MatStashScatterBegin_BTS (matstash.c:920)<br>==683913== by 0x82C009D: MatStashScatterBegin_Private (matstash.c:440)<br>==683913== by 0x73E77BF: MatAssemblyBegin_MPIAIJ (mpiaij.c:768)<br>==683913== by 0x81F0777: MatAssemblyBegin (matrix.c:5807)<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jan 27, 2026 at 6:13 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Tue, Jan 27, 2026 at 3:34 PM neil liu <<a href="mailto:liufield@gmail.com" target="_blank">liufield@gmail.com</a>> wrote:</div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Dear Petsc users and developers,<br></div><div><br></div><div>I am exploring the mapping between local, partition and global dofs.<br></div><div>The following is my pseudo code, dof2Partitionmapping denotes the mapping between the local dof (20 local dofs each tetrahedra) and partition dof.<br></div></div></blockquote><div><br></div><div>We usually say cell, local, and global dofs.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>Is this mapping determined by Petsc itself under the hood (PetscSectionGetOffset)?</div></div></blockquote><div><br></div><div>Plex just iterates over the points in the canonical numbering (cells, vertices, faces, edges). You can change the iteration order using</div><div><br></div><div> <a href="https://urldefense.us/v3/__https://petsc.org/main/manualpages/PetscSection/PetscSectionSetPermutation/__;!!G_uCfscf7eWS!axRnfOFmHcGL3iKKS0tSlsif0QKDXBdDwXWrPhWj9HPh6Td-Xx10T5eHPX4YORD_ITemssQl-KasmwS3V3kG$" target="_blank">https://petsc.org/main/manualpages/PetscSection/PetscSectionSetPermutation/</a></div><div><br></div><div>You can use that, for example, to place all ghost dofs at the end.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>For now, I am coding this mapping (local to partition) myself just based on the edge and face number in the partition. It seems the results are reasonable. But with this kind of self-defined mapping, the owned dofs and ghost dofs are interleaved. Will this bring bad results for the communication of MatStash? <br></div><div><br></div><div>Thanks,<br></div><div>Xiaodong<br></div><div><br></div><div><b>1. set 2 dofs for each edge and 2 dofs for each edge face respectively</b></div><div>PetscSectionSetChart(s, faceStart, edgeEnd);</div><div>PetscSectionSetDof(s, faceIndex, 2); </div><div>PetscSectionSetFieldDof(s, faceIndex, 0, 1);<br></div><div>PetscSectionSetDof(s, edgeIndex, 2); </div><div>PetscSectionSetFieldDof(s, edgeIndex, 0, 1);<br></div><div>PetscSectionsetup(s)</div><div><br></div><div><b>2. Create matrix based on DMPlex</b><br></div><div> DMSetMatType(dm, MATAIJ); </div><div> DMCreateMatrix(dm, &A);<br></div><div><br></div><div><b>3. loop over elements to set local values for Matrix</b><br></div><div>MatSetValuesLocal(A, dof2Partitionmapping.size(), dof2Partitionmapping.data(), dof2Partitionmapping.size(), dof2Partitionmapping.data(), Matrix_Local.data(), ADD_VALUES);</div></div>
</blockquote></div><div><br clear="all"></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!axRnfOFmHcGL3iKKS0tSlsif0QKDXBdDwXWrPhWj9HPh6Td-Xx10T5eHPX4YORD_ITemssQl-Kasm4WP45iV$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><div><br clear="all"></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!axRnfOFmHcGL3iKKS0tSlsif0QKDXBdDwXWrPhWj9HPh6Td-Xx10T5eHPX4YORD_ITemssQl-Kasm4WP45iV$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>