<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div><br></div> Handling empty diagonal entries on matrices is often problematic, just as you describe. <div><br></div><div> I suggest placing explicit zeros on the diagonal first before providing the other entries, which might be the cleanest and most efficient approach. So have each MPI rank loop over its local rows and call MatSetValue() for each diagonal entry and then continue with your other MatSetValues(). Do not call MatAssemblyBegin/End() after you have provided the zeros on the diagonal just chug straight into setting the other values.</div><div><br></div><div> Barry</div><div><br></div><div> As you observed, trying to add the zero entries in the matrix after it is assembled is terribly inefficient and not the way to go.</div><div><br></div><div> I've considered adding a matrix option to force zero entries on the diagonal, but I never completed my consideration. For example, MatSetOption(A, MAT_NONEMPTY_DIAGONAL,PETSC_TRUE); and when this option is set, MatAssemblyBegin fills up any empty diagonal entries automatically.</div><div><br></div><div><br></div><div><div><br><blockquote type="cite"><div>On Jun 27, 2025, at 6:26 AM, JAMOND Olivier <Olivier.JAMOND@cea.fr> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><div id="divtagdefaultwrapper" dir="ltr" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-size: 12pt; font-family: Calibri, Helvetica, sans-serif;"><p style="margin-top: 0px; margin-bottom: 0px;"></p><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;">Hello,</span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br><span style="font-family: "Courier New", monospace;"></span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;">I am working on a PDE solver which uses petsc to solve its sparse distributed linear systems. I am mainly dealing with MPIAIJ matrices.</span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br><span style="font-family: "Courier New", monospace;"></span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">In some situations, it may happen that the matrices considered does not have non-zero term on the diagonal. For instance I work on a case which have a stokes like saddle-point structure (in a MPIAIJ, not a MATNEST</span><span style="font-family: "Courier New", monospace;">)</span><span style="font-family: "Courier New", monospace;">:</span></span><br><span style="font-family: "Courier New", monospace;"></span><br><span style="font-family: "Courier New", monospace;"></span><span style="font-family: "Courier New", monospace;">[A Bt][U]=[F]</span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;">[B 0 ][L] [0]</span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br><span style="font-family: "Courier New", monospace;"></span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;">I d</span><span style="font-family: "Courier New", monospace;">o not insert null terms in the zero block.</span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"><br></span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;">In some cases, I use the function `</span><span style="font-size: 12pt;"><span style="font-family: "Courier New", monospace;">MatZeroRowsColumns` to handle "Dirichlet" </span><span style="font-family: "Courier New", monospace;">boundary conditions</span><span style="font-family: "Courier New", monospace;">. In this particular case, I apply Dirichlet </span><span style="font-family: "Lucida Console", Monaco, monospace;"><span style="font-family: "Courier New", monospace;">BCs only on dofs of "U". But I </span><span style="font-family: "Courier New", monospace;">get an error `</span><span style="font-family: "Courier New", monospace;">Matrix is missing diagonal entry in row X</span><span style="font-family: "Courier New", monospace;">` from the function `</span><span style="font-family: "Courier New", monospace;">MatZeroRowsColumns</span><span style="font-family: "Courier New", monospace;">`, where X is a row related to "L". </span></span></span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br><span style="font-family: "Courier New", monospace;"></span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">My first question is: is it normal that I get an error for a missing diagonal in the function </span><span style="font-family: "Courier New", monospace;">`MatZeroRowsColumns`</span><span style="font-family: "Courier New", monospace;">entry for a dof that is not involved in the list of dofs that I pass to </span></span><span style="font-family: "Courier New", monospace;">`MatZeroRowsColumns`?</span><br><span style="font-family: "Courier New", monospace;"></span><span style="font-size: 12pt; font-family: "Courier New", monospace;"><br></span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-size: 12pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">I then tried to make my code to detect that there are some missing diagonal entries, and add an explicit zero to them. My code which adds the missing diagonal entries looks like what follows. This is certainly not the best way to do that, as in my test case<span class="Apple-converted-space"> </span></span><span style="font-family: "Courier New", monospace;">about</span><span style="font-family: "Courier New", monospace;"> ~80% of the total computation time is spent in this piece of code (more precisely in `</span><span style="font-family: "Courier New", monospace;">MatSetValue(D, k, k, 0., ADD_VALUES)`</span><span style="font-family: "Courier New", monospace;">).</span><br><span style="font-family: "Courier New", monospace;">So my second question is: what would be the most efficient way to detect the missing diagonal entries, and ad explicit zeros on the diagonal at these places?</span></span></div><span style="font-size: 16px; font-family: "Courier New", monospace;"></span><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br></div><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;">Many thanks,</span><br><span style="font-family: "Courier New", monospace;">Olivier</span></div><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br></div><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> ...</span></div><p style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"></p><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY);</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY);</span></div><p style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"></p><div style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br></div><p style="margin-top: 0px; margin-bottom: 0px; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"></p><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> Mat D;</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> MatGetDiagonalBlock(A, &D);</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> PetscBool missing;</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> MatMissingDiagonal(D, &missing, NULL);</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> if (missing) {</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> IS </span><span style="font-family: "Courier New", monospace;">missingDiagEntryRows;</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> </span><span style="font-family: "Courier New", monospace;">MatFindZeroDiagonals(D, &</span><span style="font-family: "Courier New", monospace;">missingDiagEntryRows</span><span style="font-family: "Courier New", monospace;">)</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"></span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> PetscInt size;</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> ISGetLocalSize(missingDiagEntryRows, &size);</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> const PetscInt *ptr;</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> ISGetIndices(missingDiagEntryRows, &ptr);</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace; font-size: 12pt;"> for (Index i = 0; i < size; ++i) {</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> PetscInt k = ptr[i];</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> MatSetValue(D, k, k, 0., ADD_VALUES);</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> }</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> MatAssemblyBegin(D, MAT_FINAL_ASSEMBLY);</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> MatAssemblyEnd(D, MAT_FINAL_ASSEMBLY);</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><br></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> ISRestoreIndices(missingDiagEntryRows, &ptr);</span></div><div style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"><span style="font-family: "Courier New", monospace;"> }</span></div><br><p style="margin-top: 0px; margin-bottom: 0px;"></p><div style="margin-top: 0px; margin-bottom: 0px;"><br></div><div id="Signature"><div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;"><p style="margin-top: 0px; margin-bottom: 0px;"><span style="font-size: 12pt;"></span></p>_________________________________________<br><table width="450" style="color: rgb(33, 33, 33); font-family: wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif, serif, EmojiFont; font-size: 12px;"><tbody><tr><td><span style="font-size: 1.1em; color: rgb(212, 0, 0);"><b><span style="color: rgb(0, 111, 201);">Olivier Jamond</span></b></span></td></tr><tr><td style="color: rgb(0, 0, 1);"><div valign="middle"><b>Research Engineer</b> <br>French Atomic Energy and Alternative Energies Commission<br>DES/ISAS/DM2S/SEMT/DYN<br>91191 Gif sur Yvette, Cedex, France</div></td></tr><tr><td><span style="color: rgb(212, 0, 0);"><span style="color: rgb(0, 111, 201);">Email</span><span style="color: rgb(0, 111, 201);">:</span><span class="Apple-converted-space"> </span><span style=""><a href="mailto:olivier.jamond@cea.fr">olivier.jamond</a></span></span><a href="mailto:olivier.jamond@cea.fr">@cea.fr</a> <span style="color: rgb(212, 0, 0);"><span style="color: rgb(0, 111, 201);">Phone</span><span style="color: rgb(0, 111, 201);">:</span> </span>+336.78.18.18.25</td></tr></tbody></table></div></div></div></div></blockquote></div><br></div></body></html>