<div dir="ltr"><div dir="ltr">On Wed, Aug 27, 2025 at 5:15 AM Alexis SALZMAN <<a href="mailto:alexis.salzman@ec-nantes.fr">alexis.salzman@ec-nantes.fr</a>> wrote:</div><div class="gmail_quote gmail_quote_container"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div>
<p>Hello Pierre,</p>
<p>After applying your patch to my local version of PETSc, all of
the cases that previously caused the provided test to fail are now
running smoothly. In a more complex context (with more processes
and colors in my application), no errors are found and the
sub-matrices look OK.<br>
</p>
<p>Thank you very much for your time. This debugged function will
greatly simplify the development of my application.</p>
<p></p></div></blockquote><div>Thanks for taking the time to report the error in a clear fashion. We can't make the code better without this kind of cooperative effort.</div><div><br></div><div> Matt </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p>Best regards</p>
<p>A.S.<br>
</p>
<div>Le 26/08/2025 à 15:46, Matthew Knepley
a écrit :<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">On Tue, Aug 26, 2025 at 9:42 AM Pierre Jolivet
<<a href="mailto:pierre@joliv.et" target="_blank">pierre@joliv.et</a>> wrote:</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>It’s indeed very suspicious (to me) that we are using
rmap to change a column index.
<div>Switching to cmap gets your code running, but I’ll
need to see if this triggers regressions.</div>
</div>
</blockquote>
<div><br>
</div>
<div>That looks right to me. I am sure this has only been
tested for GASM, which would be symmetric.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>Thanks for the report,</div>
<div>Pierre</div>
<div><br>
</div>
<div>
<div>diff --git a/src/mat/impls/aij/mpi/mpiov.c
b/src/mat/impls/aij/mpi/mpiov.c</div>
<div>index d1037d7d817..051981ebe9a 100644</div>
<div>--- a/src/mat/impls/aij/mpi/mpiov.c</div>
<div>+++ b/src/mat/impls/aij/mpi/mpiov.c</div>
<div>@@ -2948,3 +2948,3 @@ PetscErrorCode
MatSetSeqMats_MPIAIJ(Mat C, IS rowemb, IS dcolemb, IS
ocolemb, Ma</div>
<div> </div>
<div>- PetscCall(PetscLayoutGetRange(C->rmap,
&rstart, &rend));</div>
<div>+ PetscCall(PetscLayoutGetRange(C->cmap,
&rstart, &rend));</div>
<div> shift = rend - rstart;</div>
<div><br>
</div>
<div>$ cat proc_0_output.txt</div>
<div>rstart 0 rend 4</div>
<div>Mat Object: 3 MPI processes</div>
<div> type: mpiaij</div>
<div> row 0: (0, 101.) (3, 104.) (6, 107.)
(9, 110.) </div>
<div> row 1: (2, 203.) (5, 206.) (8, 209.)
(11, 212.) </div>
<div> row 2: (1, 302.) (4, 305.) (7, 308.)
(10, 311.) </div>
<div> row 3: (0, 401.) (3, 404.) (6, 407.)
(9, 410.) </div>
<div> row 4: (2, 503.) (5, 506.) (8, 509.)
(11, 512.) </div>
<div> row 5: (1, 602.) (4, 605.) (7, 608.)
(10, 611.) </div>
<div> row 6: (0, 701.) (3, 704.) (6, 707.)
(9, 710.) </div>
<div> row 7: (2, 803.) (5, 806.) (8, 809.)
(11, 812.) </div>
<div> row 8: (1, 902.) (4, 905.) (7, 908.)
(10, 911.) </div>
<div> row 9: (0, 1001.) (3, 1004.) (6, 1007.)
(9, 1010.) </div>
<div> row 10: (2, 1103.) (5, 1106.) (8, 1109.)
(11, 1112.) </div>
<div> row 11: (1, 1202.) (4, 1205.) (7, 1208.)
(10, 1211.) </div>
<div>idxr proc</div>
<div>IS Object: 2 MPI processes</div>
<div> type: general</div>
<div>[0] Number of indices in set 4</div>
<div>[0] 0 0</div>
<div>[0] 1 1</div>
<div>[0] 2 2</div>
<div>[0] 3 3</div>
<div>[1] Number of indices in set 4</div>
<div>[1] 0 4</div>
<div>[1] 1 5</div>
<div>[1] 2 6</div>
<div>[1] 3 7</div>
<div>idxc proc</div>
<div>IS Object: 2 MPI processes</div>
<div> type: general</div>
<div>[0] Number of indices in set 2</div>
<div>[0] 0 0</div>
<div>[0] 1 1</div>
<div>[1] Number of indices in set 2</div>
<div>[1] 0 6</div>
<div>[1] 1 7</div>
<div>Mat Object: 2 MPI processes</div>
<div> type: mpiaij</div>
<div> row 0: (0, 101.) (2, 107.) </div>
<div> row 1: </div>
<div> row 2: (1, 302.) (3, 308.) </div>
<div> row 3: (0, 401.) (2, 407.) </div>
<div> row 4: </div>
<div> row 5: (1, 602.) (3, 608.) </div>
<div> row 6: (0, 701.) (2, 707.) </div>
<div> row 7: </div>
<div>rstart 0 rend 4</div>
<div>local row 0: ( 0 , 1.010000e+02) ( 2 ,
1.070000e+02)</div>
<div>local row 1:</div>
<div>local row 2: ( 1 , 3.020000e+02) ( 3 ,
3.080000e+02)</div>
<div>local row 3: ( 0 , 4.010000e+02) ( 2 ,
4.070000e+02)</div>
<div><br>
<blockquote type="cite">
<div>On 26 Aug 2025, at 3:18 PM, Pierre Jolivet <<a href="mailto:pierre@joliv.et" target="_blank">pierre@joliv.et</a>>
wrote:</div>
<br>
<div>
<div><br>
<div>
<blockquote type="cite">
<div>On 26 Aug 2025, at 12:50 PM, Alexis
SALZMAN <<a href="mailto:alexis.salzman@ec-nantes.fr" target="_blank">alexis.salzman@ec-nantes.fr</a>>
wrote:</div>
<br>
<div>
<div>
<p>Mark, you were right and I was wrong
about the dense matrix. Adding
explicit zeros to the distributed
matrix used to extract the
sub-matrices (making it dense) in my
test does not change the behaviour:
there is still an error.</p>
<p>I am finding it increasingly
difficult to understand the logic of
the row and column 'IS' creation. I
ran many tests to achieve the desired
result: a rectangular sub-matrix (so a
rectangular or square sub-matrix
appears to be possible). However, many
others resulted in the same kind of
error.</p>
</div>
</div>
</blockquote>
<div>This may be a PETSc bug
in MatSetSeqMats_MPIAIJ().</div>
<div>
<div>-> 2965<span style="white-space:pre-wrap"> </span>
PetscCall(MatSetValues(aij->B, 1,
&row, 1, &col, &v,
INSERT_VALUES));</div>
<div>col has a value of 4, which doesn’t
make sense since the output Mat has 4
columns (thus, has the error message
suggests, the value should be lower than
or equal to 3).</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Pierre</div>
</div>
<blockquote type="cite">
<div>
<div>
<p>From what I observed, the test only
works if the column selection
contribution (size_c in the test) has
a specific value related to the row
selection contribution (size_r in the
test) for proc 0 (rank for both
communicator and sub-communicator):</p>
<ul>
<li>if size_r==2 then if size_c<=2
it works.</li>
<li>if size_r>=3 and size_r<=5
then size_c==size_r is the only
working case.</li>
</ul>
<p>This occurs "regardless" of what is
requested in proc 1 and in selr/selc
(It can't be a dummy setting, though).
In any case, it's certainly not an
exhaustive analysis.<br>
</p>
<p>Many thanks to anyone who can explain
to me the logic behind the
construction of row and column 'IS'.</p>
<p>Regards</p>
<p>A.S.<br>
</p>
<p><br>
</p>
<div>Le 25/08/2025 à 20:00, Alexis
SALZMAN a écrit :<br>
</div>
<blockquote type="cite">
<p>Thanks Mark for your attention.</p>
<p>The uncleaned error message,
compared to my post in July, is as
follows:</p>
<p> <span style="font-family:monospace"><span style="background-color:rgb(255,255,255)">[0]PETSC ERROR:
--------------------- Error
Message
--------------------------------------------------------------</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: Argument out
of range</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: Column too
large: col 4 max 3</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: See <a href="https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!dWBkCu100EMuxu8ooVUnqSFN7OhzOBoNHAiwDYEQ5cJ921sU5hdFb-G24ounZFeUQgZkfWqGRX4iIHyQ-xLQElJst5RbKa2pGnk$" target="_blank">https://petsc.org/release/faq/</a>
for trouble shooting.</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: Petsc Release
Version 3.22.2, unknown </span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: subnb with 3
MPI process(es) and PETSC_ARCH
on <a href="https://urldefense.us/v3/__http://pc-str97.ec-nantes.fr__;!!G_uCfscf7eWS!eQa_exf2PCgmMQ0L4h9al-nkWsWRJJ1Zwkjm_qHJsqT0zwLzW7eMjKlkRssc6loRju6u04y4yp9L0U39POoDIvyQKcfm9nWwEJg$" target="_blank">pc-str97.ec-nantes.fr</a>
by salzman Mon Aug 25 19:11:37
2025</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: Configure
options:
PETSC_ARCH=real_fc41_Release_gcc_i4
PETSC_DIR=/home/salzman/devel/ExternalLib/build/PETSC/petsc
--doCleanup=1
--with-scalar-type=real
--known-level1-dcach</span><br>
e-linesize=64 --with-cc=gcc
--CFLAGS="-fPIC "
--CC_LINKER_FLAGS=-fopenmp
--with-cxx=g++
--with-cxx-dialect=c++20
--CXXFLAGS="-fPIC "
--CXX_LINKER_FLAGS=-fopenmp
--with-fc=gfortran --FFLAGS=<br>
"-fPIC "
--FC_LINKER_FLAGS=-fopenmp
--with-debugging=0
--with-fortran-bindings=0
--with-fortran-kernels=1
--with-mpi-compilers=0
--with-mpi-include=/usr/include/openmpi-x86_64
--with-mpi-li<br>
b="[/usr/lib64/openmpi/lib/libmpi.so,/usr/lib64/openmpi/lib/libmpi.so,/usr/lib64/openmpi/lib/libmpi_mpifh.so]"
--with-blas-lib="[/opt/intel/oneapi/mkl/latest/lib/libmkl_intel_lp64.so,/opt/i<br>
ntel/oneapi/mkl/latest/lib/libmkl_gnu_thread.so,/opt/intel/oneapi/mkl/latest/lib/libmkl_core.so]"
--with-lapack-lib="[/opt/intel/oneapi/mkl/latest/lib/libmkl_intel_lp64.so,/opt/intel/oneapi<br>
/mkl/latest/lib/libmkl_gnu_thread.so,/opt/intel/oneapi/mkl/latest/lib/libmkl_core.so]"
--with-mumps=1
--with-mumps-include=/home/salzman/local/i4_gcc/include
--with-mumps-lib="[/home/salzma<br>
n/local/i4_gcc/lib/libdmumps.so,/home/salzman/local/i4_gcc/lib/libmumps_common.so,/home/salzman/local/i4_gcc/lib/libpord.so]"
--with-scalapack-lib="[/opt/intel/oneapi/mkl/latest/lib/libmkl_<br>
scalapack_lp64.so,/opt/intel/oneapi/mkl/latest/lib/libmkl_blacs_openmpi_lp64.so]"
--with-mkl_pardiso=1
--with-mkl_pardiso-include=/opt/intel/oneapi/mkl/latest/include
--with-mkl_pardiso-lib<br>
="[/opt/intel/oneapi/mkl/latest/lib/intel64/libmkl_intel_lp64.so]"
--with-hdf5=1
--with-hdf5-include=/usr/include/openmpi-x86_64
--with-hdf5-lib="[/usr/lib64/openmpi/lib/libhdf5.so]" --with<br>
-pastix=0 --download-pastix=no
--with-hwloc=1
--with-hwloc-dir=/home/salzman/local/i4_gcc
--download-hwloc=no
--with-ptscotch-include=/home/salzman/local/i4_gcc/include
--with-ptscotch-lib=<br>
"[/home/salzman/local/i4_gcc/lib/libptscotch.a,/home/salzman/local/i4_gcc/lib/libptscotcherr.a,/home/salzman/local/i4_gcc/lib/libptscotcherrexit.a,/home/salzman/local/i4_gcc/lib/libscotch.a<br>
,/home/salzman/local/i4_gcc/lib/libscotcherr.a,/home/salzman/local/i4_gcc/lib/libscotcherrexit.a]"
--with-hypre=1
--download-hypre=yes
--with-suitesparse=1
--with-suitesparse-include=/home/<br>
salzman/local/i4_gcc/include
--with-suitesparse-lib="[/home/salzman/local/i4_gcc/lib/libsuitesparseconfig.so,/home/salzman/local/i4_gcc/lib/libumfpack.so,/home/salzman/local/i4_gcc/lib/libk<br>
lu.so,/home/salzman/local/i4_gcc/lib/libcholmod.so,/home/salzman/local/i4_gcc/lib/libspqr.so,/home/salzman/local/i4_gcc/lib/libcolamd.so,/home/salzman/local/i4_gcc/lib/libccolamd.so,/home/s<br>
alzman/local/i4_gcc/lib/libcamd.so,/home/salzman/local/i4_gcc/lib/libamd.so,/home/salzman/local/i4_gcc/lib/libmetis.so]"
--download-suitesparse=no
--with-python-exec=python3.12
--have-numpy<br>
=1 ---with-petsc4py=1
---with-petsc4py-test-np=4
---with-mpi4py=1
--prefix=/home/salzman/local/i4_gcc/real_arithmetic
COPTFLAGS="-O3 -g "
CXXOPTFLAGS="-O3 -g "
FOPTFLAGS="-O3 -g "<span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: #1
MatSetValues_SeqAIJ() at
/home/salzman/devel/PETSc/petsc/src/mat/impls/aij/seq/aij.c:426</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: #2
MatSetValues() at
/home/salzman/devel/PETSc/petsc/src/mat/interface/matrix.c:1543</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: #3
MatSetSeqMats_MPIAIJ() at
/home/salzman/devel/PETSc/petsc/src/mat/impls/aij/mpi/mpiov.c:2965</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: #4
MatCreateSubMatricesMPI_MPIXAIJ()
at
/home/salzman/devel/PETSc/petsc/src/mat/impls/aij/mpi/mpiov.c:3163</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: #5
MatCreateSubMatricesMPI_MPIAIJ()
at
/home/salzman/devel/PETSc/petsc/src/mat/impls/aij/mpi/mpiov.c:3196</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: #6
MatCreateSubMatricesMPI() at
/home/salzman/devel/PETSc/petsc/src/mat/interface/matrix.c:7293</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: #7 main() at
subnb.c:181</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR: No PETSc
Option Table entries</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">[0]PETSC ERROR:
----------------End of Error
Message -------send entire error
message to <a href="mailto:petsc-maint@mcs.anl.gov" target="_blank">petsc-maint@mcs.anl.gov</a>----------</span><span style="background-color:rgb(255,255,255)"> </span><br>
<span style="background-color:rgb(255,255,255)">--------------------------------------------------------------------------</span><span style="background-color:rgb(255,255,255)"></span></span></p>
<p>This message comes from executing
the attached test (I simplified the
test by removing the block size from
the matrix used for extraction,
compared to the July test). In
proc_xx_output.txt, you will find
the output from the code execution
with the -ok option (i.e. irow/idxr
and icol/idxc are the same, i.e. a
square sub-block for colour 0
distributed across the first two
processes). <br>
</p>
<p>Has expected in this case we obtain
the 0,3,6,9 sub-block terms, which
are distributed across processes 0
and 1 (two rows per proc).</p>
<p>When asking for rectangular
sub-block (i.e. with no option) it
crash with column to large on
process 0: 4 col max 3 ??? I ask for
4 rows and 2 columns in this process
???</p>
<p>Otherwise, I mention the dense
aspect of the matrix in ex183.c,
because, in this case, no matter
what selection is requested, all
terms are non-null. If there is an
issue with the way the selection is
coded in the user program, I think
it will be masked thanks to the full
graph representation. However, this
may not be the case — I should test
it.</p>
<p>I'll take a look at ex23.c.</p>
<p>Thanks,</p>
<p>A.S.<br>
</p>
<p><br>
</p>
<p><br>
</p>
<div>Le 25/08/2025 à 17:55, Mark Adams
a écrit :<br>
</div>
<blockquote type="cite">
<div dir="ltr">Ah, OK, never say
never.
<div><br>
</div>
<div>MatCreateSubMatrices seems to
support creating a new matrix
with the communicator of the IS.</div>
<div>It just needs to read from
the input matrix and does not
use it for communication, so it
can do that.</div>
<div><br>
</div>
<div>As far as rectangular
matrices, there is no reason not
to support that (the row IS and
column IS can be distinct).</div>
<div>Can you send the whole error
message?</div>
<div>There may not be a test that
does this,
but src/mat/tests/ex23.c looks
like it may be a
rectangular matrix output.</div>
<div><br>
</div>
<div>And, it should not matter if
the input matrix has a 100% full
sparse matrix. It is still
MatAIJ.</div>
<div>The semantics and API is the
same for sparse or dense
matrices.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Mark</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On
Mon, Aug 25, 2025 at 7:31 AM
Alexis SALZMAN <<a href="mailto:alexis.salzman@ec-nantes.fr" target="_blank">alexis.salzman@ec-nantes.fr</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> Hi,<br>
<br>
Thanks for your answer, Mark.
Perhaps
MatCreateSubMatricesMPI is the
only PETSc function that acts
on a sub-communicator — I'm
not sure — but it's clear that
there's no ambiguity on that
point. The first line of the
documentation for that
function states that it 'may
live on subcomms'. This is
confirmed by the
'src/mat/tests/ex183.c' test
case. I used this test case to
understand the function, which
helped me with my code and the
example I provided in my
initial post. Unfortunately,
in this example, the matrix
from which the sub-matrices
are extracted is dense, even
though it uses a sparse
structure. This does not
clarify how to define
sub-matrices when extracting
from a sparse distributed
matrix. Since my initial post,
I have discovered that having
more columns than rows can
also result in the same error
message.
<p>So, my questions boil down
to:</p>
<p>Can MatCreateSubMatricesMPI
extract rectangular matrices
from a square distributed
sparse matrix?</p>
<p>If not, the fact that only
square matrices can be
extracted in this context
should perhaps be mentioned
in the documentation.<br>
</p>
<p>If so, I would be very
grateful for any assistance
in defining an IS pair in
this context.<br>
</p>
<p>Regards</p>
<p>A.S.<br>
</p>
<div>Le 27/07/2025 à 00:15,
Mark Adams a écrit :<br>
</div>
<blockquote type="cite">
<div dir="ltr">First, you
can not mix communicators
in PETSc calls in general
(ever?), but this error
looks like you might
be asking for a row from
the matrix that does not
exist.
<div>You should start with
a PETSc example code.
Test it and modify it to
suit your needs.</div>
<div><br>
</div>
<div>Good luck,</div>
<div>Mark</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On
Fri, Jul 25, 2025 at
9:31 AM Alexis SALZMAN
<<a href="mailto:alexis.salzman@ec-nantes.fr" target="_blank">alexis.salzman@ec-nantes.fr</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
As I am relatively new
to Petsc, I may have
misunderstood how to use
the <br>
MatCreateSubMatricesMPI
function. The attached
code is tuned for three
<br>
processes and extracts
one matrix for each
colour of a
subcommunicator <br>
that has been created
using the MPI_Comm_split
function from an MPIAij
<br>
matrix. The following
error message appears
when the code is set to
its <br>
default configuration
(i.e. when a rectangular
matrix is extracted with
<br>
more rows than columns
for colour 0):<br>
<br>
[0]PETSC ERROR:
---------------------
Error Message <br>
--------------------------------------------------------------<br>
[0]PETSC ERROR: Argument
out of range<br>
[0]PETSC ERROR: Column
too large: col 4 max 3<br>
[0]PETSC ERROR: See <a href="https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!ZqH097BZ0G0O3WI7RWrwIKFNpyk0czSWEqfusAeTlgEygAffwpgBUzsLw1TIoGkjZ3mYG-NRQxxFoxU4y8EyY0ofiz9I43Qwe0w$" rel="noreferrer" target="_blank">https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!ZqH097BZ0G0O3WI7RWrwIKFNpyk0czSWEqfusAeTlgEygAffwpgBUzsLw1TIoGkjZ3mYG-NRQxxFoxU4y8EyY0ofiz9I43Qwe0w$</a>
for trouble shooting.<br>
[0]PETSC ERROR: Petsc
Release Version 3.22.2,
unknown<br>
<br>
... petsc git hash
2a89477b25f compiled on
a dell i9 computer with
Gcc <br>
14.3, mkl 2025.2, .....<br>
[0]PETSC ERROR: #1
MatSetValues_SeqAIJ() at
<br>
...petsc/src/mat/impls/aij/seq/aij.c:426<br>
[0]PETSC ERROR: #2
MatSetValues() at <br>
...petsc/src/mat/interface/matrix.c:1543<br>
[0]PETSC ERROR: #3
MatSetSeqMats_MPIAIJ()
at <br>
.../petsc/src/mat/impls/aij/mpi/mpiov.c:2965<br>
[0]PETSC ERROR: #4
MatCreateSubMatricesMPI_MPIXAIJ()
at <br>
.../petsc/src/mat/impls/aij/mpi/mpiov.c:3163<br>
[0]PETSC ERROR: #5
MatCreateSubMatricesMPI_MPIAIJ()
at <br>
.../petsc/src/mat/impls/aij/mpi/mpiov.c:3196<br>
[0]PETSC ERROR: #6
MatCreateSubMatricesMPI()
at <br>
.../petsc/src/mat/interface/matrix.c:7293<br>
[0]PETSC ERROR: #7
main() at sub.c:169<br>
<br>
When the '-ok' option is
selected, the code
extracts a square matrix
for <br>
colour 0, which runs
smoothly in this case.
Selecting the '-trans' <br>
option swaps the row and
column selection
indices, providing a <br>
transposed submatrix
smoothly. For colour 1,
which uses only one
process <br>
and is therefore
sequential, rectangular
extraction is OK
regardless of <br>
the shape.<br>
<br>
Is this dependency on
the shape expected? Have
I missed an important <br>
tuning step somewhere?<br>
<br>
Thank you in advance for
any clarification.<br>
<br>
Regards<br>
<br>
A.S.<br>
<br>
P.S.: I'm sorry, but as
I'm leaving my office
for the following weeks
<br>
this evening, I won't be
very responsive during
this period.<br>
<br>
<br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<div><br clear="all">
</div>
<div><br>
</div>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>What most experimenters take for granted before
they begin their experiments is infinitely more
interesting than any results to which their
experiments lead.<br>
-- Norbert Wiener</div>
<div><br>
</div>
<div><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eQa_exf2PCgmMQ0L4h9al-nkWsWRJJ1Zwkjm_qHJsqT0zwLzW7eMjKlkRssc6loRju6u04y4yp9L0U39POoDIvyQKcfmEYm5G4Y$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote></div><div><br clear="all"></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eHMhc5L1tNjyBrEpH0recWwhp5pSwBnZuh09zH8p-ZQyM8m3kxz8ryLVwi7nXr0NNKedHWcHfY5etM4RQkAm$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>