<div dir="ltr"><font size="4">Thanks a lot, Matt. </font><div><p class="gmail-isSelectedEnd"><font size="4">I assemble a matrix using <code>MatSetValuesLocal()</code>, where the cell-to-local mapping comes from <code>PetscSectionGetOffset()</code>.</font></p><p class="gmail-isSelectedEnd"><font size="4">With 2 MPI ranks, Valgrind (rank 0) reports uninitialized data being passed to <code>MPI_Isend</code> during matrix assembly (inside <code>matstash.c</code>).</font></p><p class="gmail-isSelectedEnd"><font size="4">Two observations:</font></p><ul><li><p class="gmail-isSelectedEnd"><font size="4">If I hard-code the local mapping to always be <code>1</code>, the Valgrind warning disappears.</font></p></li><li><p class="gmail-isSelectedEnd"><font size="4">The warning occurs with PETSc <strong>3.23.3</strong>, but disappears when switching to <strong>3.24.3</strong> under the same setup.</font></p></li></ul><p class="gmail-isSelectedEnd"><font size="4">This suggests either uninitialized local indices/values on my side, or a change in PETSc between these versions affecting stash packing or index handling.</font></p><p><font size="4">Does this type of Valgrind warning usually point to user-provided local indices being uninitialized, and were there relevant changes in this area between 3.23.3 and 3.24.x?</font></p></div><div><span style="font-size:large"><br></span></div><div><span style="font-size:large">Thanks, </span></div><div><span style="font-size:large">Xiaodong </span></div><div><font size="4"><br></font></div><div>==683913== Syscall param write(buf) points to uninitialised byte(s)<br>==683913== at 0xCAD15A8: write (in /usr/lib64/<a href="https://urldefense.us/v3/__http://libc-2.28.so__;!!G_uCfscf7eWS!Z7m5oJFaTt_KCIUQJxJx-4SEoHIv5k1rWuDb3ClOur8MISuVMaOzs6WTo6lqZ2OWUBvBAKwK4hyKszJ-HoZ8hA$" target="_blank">libc-2.28.so</a>)<br>==683913== by 0xBB9B0A9: MPIDI_CH3I_Sock_write (sock.c:2622)<br>==683913== by 0xBBA12DF: MPIDI_CH3_iStartMsg (ch3_istartmsg.c:68)<br>==683913== by 0xBB55F82: MPIDI_CH3_RndvSend (ch3u_rndv.c:48)<br>==683913== by 0xBB70905: MPID_Isend (mpid_isend.c:159)<br>==683913== by 0xB83EDB7: internal_Isend_c (isend.c:273)<br>==683913== by 0xB83EFE9: PMPI_Isend_c (isend.c:363)<br>==683913== by 0x82CAC1A: MatStashBTSSend_Private (matstash.c:793)<br>==683913== by 0x6AA2BF8: PetscCommBuildTwoSidedFReq_Reference (mpits.c:311)<br>==683913== by 0x6AA9DAD: PetscCommBuildTwoSidedFReq (mpits.c:515)<br>==683913== by 0x82CEB46: MatStashScatterBegin_BTS (matstash.c:920)<br>==683913== by 0x82C009D: MatStashScatterBegin_Private (matstash.c:440)<br>==683913== by 0x73E77BF: MatAssemblyBegin_MPIAIJ (mpiaij.c:768)<br>==683913== by 0x81F0777: MatAssemblyBegin (matrix.c:5807)<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jan 27, 2026 at 6:13 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Tue, Jan 27, 2026 at 3:34 PM neil liu <<a href="mailto:liufield@gmail.com" target="_blank">liufield@gmail.com</a>> wrote:</div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Dear Petsc users and developers,<br></div><div><br></div><div>I am exploring the mapping between local, partition and global dofs.<br></div><div>The following is my pseudo code, dof2Partitionmapping denotes the mapping between the local dof (20 local dofs each tetrahedra) and partition dof.<br></div></div></blockquote><div><br></div><div>We usually say cell, local, and global dofs.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>Is this mapping determined by Petsc itself under the hood (PetscSectionGetOffset)?</div></div></blockquote><div><br></div><div>Plex just iterates over the points in the canonical numbering (cells, vertices, faces, edges). You can change the iteration order using</div><div><br></div><div> <a href="https://urldefense.us/v3/__https://petsc.org/main/manualpages/PetscSection/PetscSectionSetPermutation/__;!!G_uCfscf7eWS!Z7m5oJFaTt_KCIUQJxJx-4SEoHIv5k1rWuDb3ClOur8MISuVMaOzs6WTo6lqZ2OWUBvBAKwK4hyKszJ1HUAjOw$" target="_blank">https://petsc.org/main/manualpages/PetscSection/PetscSectionSetPermutation/</a></div><div><br></div><div>You can use that, for example, to place all ghost dofs at the end.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>For now, I am coding this mapping (local to partition) myself just based on the edge and face number in the partition. It seems the results are reasonable. But with this kind of self-defined mapping, the owned dofs and ghost dofs are interleaved. Will this bring bad results for the communication of MatStash? <br></div><div><br></div><div>Thanks,<br></div><div>Xiaodong<br></div><div><br></div><div><b>1. set 2 dofs for each edge and 2 dofs for each edge face respectively</b></div><div>PetscSectionSetChart(s, faceStart, edgeEnd);</div><div>PetscSectionSetDof(s, faceIndex, 2); </div><div>PetscSectionSetFieldDof(s, faceIndex, 0, 1);<br></div><div>PetscSectionSetDof(s, edgeIndex, 2); </div><div>PetscSectionSetFieldDof(s, edgeIndex, 0, 1);<br></div><div>PetscSectionsetup(s)</div><div><br></div><div><b>2. Create matrix based on DMPlex</b><br></div><div> DMSetMatType(dm, MATAIJ); </div><div> DMCreateMatrix(dm, &A);<br></div><div><br></div><div><b>3. loop over elements to set local values for Matrix</b><br></div><div>MatSetValuesLocal(A, dof2Partitionmapping.size(), dof2Partitionmapping.data(), dof2Partitionmapping.size(), dof2Partitionmapping.data(), Matrix_Local.data(), ADD_VALUES);</div></div>
</blockquote></div><div><br clear="all"></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Z7m5oJFaTt_KCIUQJxJx-4SEoHIv5k1rWuDb3ClOur8MISuVMaOzs6WTo6lqZ2OWUBvBAKwK4hyKszK-GUW0Jw$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>