<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div>MatMPIAIJSetPreallocation(A, 3, NULL, 0, NULL);<div class=""><br class=""></div><div class="">The 0 indicates you expect that there will be no matrix entries in the columns of the matrix that represent degrees of freedom that are not stored on the given rank. In your case this means you expect the matrix to be block diagonal with one block per MPI rank. You need to put an appropriate value instead of 1. If you matrix is Tri-diagonal then using 1 will be enough. But if your matrix has a more complicated structure (but with nonzeros on the diagonal) then you need to use 2.</div><div class=""><br class=""></div><div class=""> Barry</div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Aug 16, 2022, at 1:45 PM, Patrick Alken <<a href="mailto:patrick.alken@geomag.info" class="">patrick.alken@geomag.info</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
<div class=""><p class="">I have managed to simplify my example program, and removed the
dependencies on slepc and gsl (attached). <br class="">
</p><p class="">The program defines a 800x800 matrix with 3 non-zero entries per
row, which are all set to 1.0.</p><p class="">When I run the program on 1 processor, everything works
correctly. When I run it on 2+ processors, I get the following
error:</p><p class="">-----</p><p class="">first = 400 last = 800 <br class="">
[1]PETSC ERROR: --------------------- Error Message
-------------------------------------------------------------- <br class="">
[1]PETSC ERROR: Argument out of range <br class="">
[1]PETSC ERROR: New nonzero at (0,399) caused a malloc <br class="">
Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE)
to turn off this check <br class="">
[1]PETSC ERROR: See <a class="moz-txt-link-freetext" href="https://petsc.org/release/faq/">https://petsc.org/release/faq/</a> for trouble
shooting. <br class="">
[1]PETSC ERROR: Petsc Release Version 3.17.4, unknown <br class="">
[1]PETSC ERROR: Configure options --download-make --download-mpich
--download-scalapack --download-cmake --download-mumps <br class="">
[1]PETSC ERROR: #1 MatSetValues_MPIAIJ() at
/data/palken/petsc/src/mat/impls/aij/mpi/mpiaij.c:506 <br class="">
[1]PETSC ERROR: #2 MatSetValues() at
/data/palken/petsc/src/mat/interface/matrix.c:1343 <br class="">
first = 0 last = 400 <br class="">
[0]PETSC ERROR: --------------------- Error Message
-------------------------------------------------------------- <br class="">
[0]PETSC ERROR: Argument out of range <br class="">
[0]PETSC ERROR: New nonzero at (399,400) caused a malloc <br class="">
Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE)
to turn off this check <br class="">
[0]PETSC ERROR: See <a class="moz-txt-link-freetext" href="https://petsc.org/release/faq/">https://petsc.org/release/faq/</a> for trouble
shooting. <br class="">
[0]PETSC ERROR: Petsc Release Version 3.17.4, unknown <br class="">
[0]PETSC ERROR: Configure options --download-make --download-mpich
--download-scalapack --download-cmake --download-mumps <br class="">
[0]PETSC ERROR: #1 MatSetValues_MPIAIJ() at
/data/palken/petsc/src/mat/impls/aij/mpi/mpiaij.c:506 <br class="">
[0]PETSC ERROR: #2 MatSetValues() at
/data/palken/petsc/src/mat/interface/matrix.c:1343<br class="">
</p><p class="">-----</p><p class="">As far as I understand, processor 1 will have rows 1-400, while
processor 2 will have rows 401-800. From my reading of the
documentation, each processor will have all columns (1-800) for
their block of rows. So I am at a loss to understand why the
program is generating these errors. Any help is appreciated.</p><p class="">Patrick<br class="">
</p>
<div class="moz-cite-prefix">On 8/15/22 04:29, Mark Adams wrote:<br class="">
</div>
<blockquote type="cite" cite="mid:CADOhEh5PQ=G+xkkP9k11sHU7QQYWJhgMXkT4zvXM7u5-Oo7UKQ@mail.gmail.com" class="">
<meta http-equiv="content-type" content="text/html; charset=UTF-8" class="">
<div dir="ltr" class="">You might need to fix this:
<div class=""><br class="">
</div>
<div class="">
<pre class="gmail-aLF-aPX-K0-aPE" style="font-family: "Courier New", Courier, monospace, arial, sans-serif; margin-top: 0px; margin-bottom: 0px; white-space: pre-wrap; font-size: 14px;">if (i - j < m)</pre>
<pre class="gmail-aLF-aPX-K0-aPE" style="font-family: "Courier New", Courier, monospace, arial, sans-serif; margin-top: 0px; margin-bottom: 0px; white-space: pre-wrap; font-size: 14px;"></pre>
<pre class="gmail-aLF-aPX-K0-aPE" style="font-family: "Courier New", Courier, monospace, arial, sans-serif; margin-top: 0px; margin-bottom: 0px; white-space: pre-wrap; font-size: 14px;">to </pre>
<pre class="gmail-aLF-aPX-K0-aPE" style="font-family: "Courier New", Courier, monospace, arial, sans-serif; margin-top: 0px; margin-bottom: 0px; white-space: pre-wrap; font-size: 14px;"></pre>
<pre class="gmail-aLF-aPX-K0-aPE" style="font-family: "Courier New", Courier, monospace, arial, sans-serif; margin-top: 0px; margin-bottom: 0px; white-space: pre-wrap; font-size: 14px;"><pre class="gmail-aLF-aPX-K0-aPE" style="font-family:"Courier New",Courier,monospace,arial,sans-serif;margin-top:0px;margin-bottom:0px;white-space:pre-wrap">if (i - j < m && i - j >= 0)</pre><pre class="gmail-aLF-aPX-K0-aPE" style="font-family:"Courier New",Courier,monospace,arial,sans-serif;margin-top:0px;margin-bottom:0px;white-space:pre-wrap"></pre><pre class="gmail-aLF-aPX-K0-aPE" style="font-family:"Courier New",Courier,monospace,arial,sans-serif;margin-top:0px;margin-bottom:0px;white-space:pre-wrap">Mark</pre></pre>
</div>
</div>
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Aug 15, 2022 at 12:12
AM Patrick Alken <<a href="mailto:patrick.alken@geomag.info" moz-do-not-send="true" class="moz-txt-link-freetext">patrick.alken@geomag.info</a>>
wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I
am trying to assemble a matrix in parallel in Petsc, and it is
working <br class="">
for 1 processor, but when I try running it with 2 processors I
get <br class="">
errors like the following:<br class="">
<br class="">
----<br class="">
<br class="">
[1]PETSC ERROR: --------------------- Error Message <br class="">
--------------------------------------------------------------<br class="">
[1]PETSC ERROR: Argument out of range<br class="">
[1]PETSC ERROR: New nonzero at (0,207) caused a malloc<br class="">
Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR,
PETSC_FALSE) to turn <br class="">
off this check<br class="">
[1]PETSC ERROR: See <a href="https://petsc.org/release/faq/" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://petsc.org/release/faq/</a>
for trouble shooting.<br class="">
[1]PETSC ERROR: Petsc Release Version 3.17.4, unknown<br class="">
[1]PETSC ERROR: #1 MatSetValues_MPIAIJ() at <br class="">
/data/palken/petsc/src/mat/impls/aij/mpi/mpiaij.c:506<br class="">
<br class="">
----<br class="">
<br class="">
and then many MatSetValues errors follow.<br class="">
<br class="">
I have attached a minimal working example and a Makefile. My
code does <br class="">
depend on the GSL library also. I think there must be a
problem in how I <br class="">
set up the d_nnz and o_nnz arrays for the
MatMPIAIJSetPreallocation() <br class="">
function (lines 211-235 of mwe.c).<br class="">
<br class="">
I am building a matrix using radial basis functions with
compact <br class="">
support, so the matrix entries are zero when ||Xi - Xj|| >=
c, where c <br class="">
is the RBF shape parameter. This is accounted for on line 224
and 254 of <br class="">
the code. As far as I can tell I am consistent in the way I
define the <br class="">
nnz arrays and the way I call MatSetValues, so I don't know
why Petsc <br class="">
would need to malloc additional memory during the assembly
process.<br class="">
<br class="">
Any pointers would be greatly appreciated.<br class="">
<br class="">
Patrick<br class="">
</blockquote>
</div>
</blockquote>
</div>
<span id="cid:276D57E6-F820-4F08-9446-95EF31CD4981"><mwe.c></span><span id="cid:AD2A8737-9CE1-4DAB-A825-C09ED714BD84"><Makefile.txt></span></div></blockquote></div><br class=""></div></body></html>