<html><head><meta http-equiv="content-type" content="text/html; charset=us-ascii"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div><br></div> Yeah, this is silly. The check is just a "sanity-check" on the data in the file. We store redundant information in the matrix header in the file, header[3] is the total number of nonzeros in the matrix. When nz is too large, the correct value cannot fit in the header.<div><br></div><div> Changing the file format would be a breaking change meaning old files could no longer be read (MatLoad() can still load binary files saved in 1995) which would be bad.</div><div><br></div><div> I think the easiest way to handle this is to provide an option to avoid the unneeded (and doesn't work) sanity check. In other words, something like -mat_load_ignore_nz turns off the sanity check in MatLoad_MPIAIJ_Binary(). Is that something you could try in an MR or would you prefer we do it?</div><div><br></div><div> Barry</div><div><br><div><br></div><div><div><br></div><div><div><div><br><blockquote type="cite"><div>On May 17, 2023, at 8:55 AM, Fleischli Benno HSLU T&A <benno.fleischli@hslu.ch> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><div class="elementToProof" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">Dear PETSc developers<br></div><div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><br></div><div class="elementToProof ContentPasted0" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;">I am creating a very large parallel sparse matrix (MATMPIAIJ) with PETSc. I write this matrix to disk.<br></div><div class="elementToProof ContentPasted0" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;">The number of non-zeros exceeds the maximum number a 32-bit integer can hold.</div><div class="elementToProof ContentPasted0" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;">When I read the matrix from disk i get an error because there was an overflow in the nz number.</div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;">(see petsc-3.18.4/src/mat/impls/aij/seq/aij.c:4977)</div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><br></div><div class="elementToProof ContentPasted0 ContentPasted1 ContentPasted3" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;">Obviously I could compile PETSc with 64bit integers (--with-64-bit-indices).</div><div class="elementToProof ContentPasted0 ContentPasted1 ContentPasted3" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;">But I wanted to ask if there is another way. Because the total number of nonzeros nz is the only numer that exceeds the 32bit limit.</div><div class="elementToProof ContentPasted0 ContentPasted1 ContentPasted3" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;">It would not be efficient to use 64bit integers everywhere just because of this single number.<br></div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><br></div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><span class="ContentPasted0 ContentPasted1 ContentPasted4" style="font-size: 10pt;">This how I configured PETSc:</span><div class="ContentPasted0 ContentPasted1" style="font-size: 10pt;"><br class="ContentPasted4"></div><div class="ContentPasted0 ContentPasted1" style="font-size: 10pt;"><span><code spellcheck="false" class="listtype-code listindent1 list-code1 lang-typescript"><span class="author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90za3mz80zidoqhb5z69zofz80zz68zz122zez65zn3o3z89zz85z5z67zz88zz87zyz72z ContentPasted2 ContentPasted4">./configure --download-fblaslapack --download-hpddm --download-hypre --with-debugging=0<span class="Apple-converted-space"> </span><br class="ContentPasted4"></span></code></span></div><div class="ContentPasted0 ContentPasted1" style="font-size: 10pt;"><span><code spellcheck="false" class="listtype-code listindent1 list-code1 lang-typescript"><span class="author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90za3mz80zidoqhb5z69zofz80zz68zz122zez65zn3o3z89zz85z5z67zz88zz87zyz72z ContentPasted2 ContentPasted4">COPTFLAGS='-O3 -march=native -mtune=native' CXXOPTFLAGS='-O3 -march=native -mtune=native'<span class="Apple-converted-space"> </span><br class="ContentPasted4"></span></code></span></div><div class="ContentPasted0 ContentPasted1" style="font-size: 10pt;"><span><code spellcheck="false" class="listtype-code listindent1 list-code1 lang-typescript"><span class="author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90za3mz80zidoqhb5z69zofz80zz68zz122zez65zn3o3z89zz85z5z67zz88zz87zyz72z ContentPasted2 ContentPasted4">FOPTFLAGS='-O3 -march=native -mtune=native' --with-scalar-type=real (--with-mpi-dir=/home/benno/Libraries/openMPI)</span></code></span><br class="ContentPasted4"></div><span class="ContentPasted0" style="font-size: 10pt;"></span><br></div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><br></div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><br></div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;">Kind Regards</div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><br></div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;">Benno</div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><br></div><div class="elementToProof ContentPasted0 ContentPasted1" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><br></div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="font-family: Verdana, Geneva, sans-serif; font-size: 10pt;"><br></div><div id="Signature"><div><div><div><div><div><div><div style="margin: 0px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="de" style="font-size: 10pt; font-family: Tahoma, sans-serif; color: rgb(31, 73, 125);">________________________________</span><span lang="de" style="font-size: 10pt; font-family: Tahoma, sans-serif;"></span></div><div style="margin: 0px; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span lang="de" style="font-size: 9pt; font-family: Verdana, sans-serif; color: rgb(31, 73, 125);">Hochschule Luzern<br>Technik & Architektur</span></b><span lang="de" style="font-size: 9pt; font-family: Verdana, sans-serif; color: rgb(31, 73, 125);"><br></span><span lang="de" style="font-size: 9pt; font-family: Verdana, sans-serif; color: rgb(31, 73, 125);">Institute for Mechanical Engineering and Energy Technology</span></div><div style="margin: 0px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="de" style="font-size: 9pt; font-family: Verdana, sans-serif; color: rgb(31, 73, 125);">Competence Center Fluid Mechanics and Numerical Methods<b><br></b></span></div><div style="margin: 0px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="de" style="font-size: 9pt; font-family: Verdana, sans-serif; color: rgb(31, 73, 125);"><b><br></b></span></div><div style="margin: 0px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="de" style="font-size: 9pt; font-family: Verdana, sans-serif; color: rgb(31, 73, 125);"><b>Benno Fleischli</b><br>MSc in Mechanical Engineering / BSc in Electrical Engineering</span><span lang="de" style="font-size: 10pt; font-family: Tahoma, sans-serif;"></span></div><div style="margin: 0px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="de" style="font-size: 9pt; font-family: Verdana, sans-serif; color: rgb(31, 73, 125);">Wissenschaftlicher Mitarbeiter</span><span lang="de" style="font-size: 10pt; font-family: Tahoma, sans-serif;"></span><span lang="de" style="font-size: 9pt; font-family: Tahoma, sans-serif; color: rgb(31, 73, 125);"><br></span></div><div style="margin: 0px; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Verdana, sans-serif; color: rgb(31, 73, 125);"><a href="https://webmail.hslu.ch/owa/redir.aspx?C=rM5K5xPd_kKJmyojcJXaG18BaXM9wM8IqSslYJQen5H6HtdiNS2KtWNFL89yfRdRtW-Oa6xCXTk.&URL=mailto%3akatrina.kemp%40hslu.ch" target="_blank" rel="noopener noreferrer" data-auth="NotApplicable" data-safelink="true" data-linkindex="1"><span lang="de">benno.fleischli@hslu.ch</span></a></span></div></div></div></div></div></div></div></div></div></div></blockquote></div><br></div></div></div></div></body></html>