[petsc-users] MatCreateSubMatricesMPI strange behavior

[Mark Adams]

Ah, OK, never say never.

MatCreateSubMatrices seems to support creating a new matrix with the
communicator of the IS.
It just needs to read from the input matrix and does not use it for
communication, so it can do that.

As far as rectangular matrices, there is no reason not to support that (the
row IS and column IS can be distinct).
Can you send the whole error message?
There may not be a test that does this, but src/mat/tests/ex23.c looks like
it may be a rectangular matrix output.

And, it should not matter if the input matrix has a 100% full sparse
matrix. It is still MatAIJ.
The semantics and API is the same for sparse or dense matrices.

Thanks,
Mark

On Mon, Aug 25, 2025 at 7:31 AM Alexis SALZMAN <alexis.salzman at ec-nantes.fr>
wrote:

> Hi,
>
> Thanks for your answer, Mark. Perhaps MatCreateSubMatricesMPI is the only
> PETSc function that acts on a sub-communicator — I'm not sure — but it's
> clear that there's no ambiguity on that point. The first line of the
> documentation for that function states that it 'may live on subcomms'. This
> is confirmed by the 'src/mat/tests/ex183.c' test case. I used this test
> case to understand the function, which helped me with my code and the
> example I provided in my initial post. Unfortunately, in this example, the
> matrix from which the sub-matrices are extracted is dense, even though it
> uses a sparse structure. This does not clarify how to define sub-matrices
> when extracting from a sparse distributed matrix. Since my initial post, I
> have discovered that having more columns than rows can also result in the
> same error message.
>
> So, my questions boil down to:
>
> Can MatCreateSubMatricesMPI extract rectangular matrices from a square
> distributed sparse matrix?
>
> If not, the fact that only square matrices can be extracted in this
> context should perhaps be mentioned in the documentation.
>
> If so, I would be very grateful for any assistance in defining an IS pair
> in this context.
>
> Regards
>
> A.S.
> Le 27/07/2025 à 00:15, Mark Adams a écrit :
>
> First, you can not mix communicators in PETSc calls in general (ever?),
> but this error looks like you might be asking for a row from the matrix
> that does not exist.
> You should start with a PETSc example code. Test it and modify it to suit
> your needs.
>
> Good luck,
> Mark
>
> On Fri, Jul 25, 2025 at 9:31 AM Alexis SALZMAN <
> alexis.salzman at ec-nantes.fr> wrote:
>
>> Hi,
>>
>> As I am relatively new to Petsc, I may have misunderstood how to use the
>> MatCreateSubMatricesMPI function. The attached code is tuned for three
>> processes and extracts one matrix for each colour of a subcommunicator
>> that has been created using the MPI_Comm_split function from an  MPIAij
>> matrix. The following error message appears when the code is set to its
>> default configuration (i.e. when a rectangular matrix is extracted with
>> more rows than columns for colour 0):
>>
>> [0]PETSC ERROR: --------------------- Error Message
>> --------------------------------------------------------------
>> [0]PETSC ERROR: Argument out of range
>> [0]PETSC ERROR: Column too large: col 4 max 3
>> [0]PETSC ERROR: See
>> https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!ZqH097BZ0G0O3WI7RWrwIKFNpyk0czSWEqfusAeTlgEygAffwpgBUzsLw1TIoGkjZ3mYG-NRQxxFoxU4y8EyY0ofiz9I43Qwe0w$
>> for trouble shooting.
>> [0]PETSC ERROR: Petsc Release Version 3.22.2, unknown
>>
>> ... petsc git hash 2a89477b25f compiled on a dell i9 computer with Gcc
>> 14.3, mkl 2025.2, .....
>> [0]PETSC ERROR: #1 MatSetValues_SeqAIJ() at
>> ...petsc/src/mat/impls/aij/seq/aij.c:426
>> [0]PETSC ERROR: #2 MatSetValues() at
>> ...petsc/src/mat/interface/matrix.c:1543
>> [0]PETSC ERROR: #3 MatSetSeqMats_MPIAIJ() at
>> .../petsc/src/mat/impls/aij/mpi/mpiov.c:2965
>> [0]PETSC ERROR: #4 MatCreateSubMatricesMPI_MPIXAIJ() at
>> .../petsc/src/mat/impls/aij/mpi/mpiov.c:3163
>> [0]PETSC ERROR: #5 MatCreateSubMatricesMPI_MPIAIJ() at
>> .../petsc/src/mat/impls/aij/mpi/mpiov.c:3196
>> [0]PETSC ERROR: #6 MatCreateSubMatricesMPI() at
>> .../petsc/src/mat/interface/matrix.c:7293
>> [0]PETSC ERROR: #7 main() at sub.c:169
>>
>> When the '-ok' option is selected, the code extracts a square matrix for
>> colour 0, which runs smoothly in this case. Selecting the '-trans'
>> option swaps the row and column selection indices, providing a
>> transposed submatrix smoothly. For colour 1, which uses only one process
>> and is therefore sequential, rectangular extraction is OK regardless of
>> the shape.
>>
>> Is this dependency on the shape expected? Have I missed an important
>> tuning step somewhere?
>>
>> Thank you in advance for any clarification.
>>
>> Regards
>>
>> A.S.
>>
>> P.S.: I'm sorry, but as I'm leaving my office for the following weeks
>> this evening, I won't be very responsive during this period.
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20250825/12fb37e8/attachment.html>