[petsc-users] MatGetRow for global rows of a parallel matrix
Eda Oktay
eda.oktay at metu.edu.tr
Wed Jul 1 07:27:52 CDT 2020
I got it, thanks so much!
Eda
On Wed, Jul 1, 2020, 2:42 PM Matthew Knepley <knepley at gmail.com> wrote:
> On Wed, Jul 1, 2020 at 7:32 AM Eda Oktay <eda.oktay at metu.edu.tr> wrote:
>
>> I'm sorry but I still can't understand how to put an input as these
>> parameters. It is the first time I saw 'const' for IS. If it doesn't
>> refer IS, then as a parameter for row and column indices, how should I
>> give them as a valid input? It still should be an IS, right?
>>
>> Lastly, I tried to use in the form:
>>
>> IS idUi;
>> ISCreateStride(PETSC_COMM_WORLD,siz,0,1,&idUi);
>> Mat *submat;
>> MatCreateSubMatrices(U,nev,&idUi,&idUi,MAT_INITIAL_MATRIX,&submat);
>>
>
> You gave n = 4 (you call it nev), but you only have 1 rowIS. You are
> supposed to give an array of 4.
>
> Thanks,
>
> Matt
>
>
>> where nev = 4, siz = 72 and U is 72*4 matrix.
>>
>> But then, the error becomes about the matrix, U:
>>
>> [0]PETSC ERROR: --------------------- Error Message
>> --------------------------------------------------------------
>> [0]PETSC ERROR: [1]PETSC ERROR: --------------------- Error Message
>> --------------------------------------------------------------
>> [1]PETSC ERROR: Invalid argument
>> [1]PETSC ERROR: Wrong type of object: Parameter # 1
>> Invalid argument
>> [0]PETSC ERROR: Wrong type of object: Parameter # 1
>> [0]PETSC ERROR: See
>> https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>> shooting.
>> [0]PETSC ERROR: Petsc Release Version 3.13.2, Jun 02, 2020
>> [0]PETSC ERROR: ./deneme_new_vecscatter_arastep on a
>> arch-linux2-c-debug named b342.wls.metu.edu.tr by edaoktay Wed Jul 1
>> 14:25:00 2020
>> [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++
>> --download-mpich --download-openblas --download-slepc --download-metis
>> --download-parmetis --download-chaco --with-X=1
>> [1]PETSC ERROR: See
>> https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>> shooting.
>> [1]PETSC ERROR: Petsc Release Version 3.13.2, Jun 02, 2020
>> [1]PETSC ERROR: [0]PETSC ERROR: #1 ISSorted() line 1777 in
>> /home/edaoktay/petsc-3.13.2/src/vec/is/is/interface/index.c
>> [0]PETSC ERROR: ./deneme_new_vecscatter_arastep on a
>> arch-linux2-c-debug named b342.wls.metu.edu.tr by edaoktay Wed Jul 1
>> 14:25:00 2020
>> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++
>> --download-mpich --download-openblas --download-slepc --download-metis
>> --download-parmetis --download-chaco --with-X=1
>> [1]PETSC ERROR: #2 MatCreateSubMatrices_MPIDense_Local() line 104 in
>> /home/edaoktay/petsc-3.13.2/src/mat/impls/dense/mpi/mmdense.c
>> [0]PETSC ERROR: #3 MatCreateSubMatrices_MPIDense() line 66 in
>> /home/edaoktay/petsc-3.13.2/src/mat/impls/dense/mpi/mmdense.c
>> #1 ISSorted() line 1777 in
>> /home/edaoktay/petsc-3.13.2/src/vec/is/is/interface/index.c
>> [1]PETSC ERROR: #2 MatCreateSubMatrices_MPIDense_Local() line 104 in
>> /home/edaoktay/petsc-3.13.2/src/mat/impls/dense/mpi/mmdense.c
>> [0]PETSC ERROR: #4 MatCreateSubMatrices() line 6758 in
>> /home/edaoktay/petsc-3.13.2/src/mat/interface/matrix.c
>> [1]PETSC ERROR: #3 MatCreateSubMatrices_MPIDense() line 66 in
>> /home/edaoktay/petsc-3.13.2/src/mat/impls/dense/mpi/mmdense.c
>> [1]PETSC ERROR: #4 MatCreateSubMatrices() line 6758 in
>> /home/edaoktay/petsc-3.13.2/src/mat/interface/matrix.c
>>
>> Before I used &idUi, I tried to use just idUi (without & sign) and it
>> didn't work, so I decided to try using this way, since may be 'const'
>> refers this one. But how can my matrix be of wrong type? It is MPI
>> Dense.
>>
>> Thanks!
>>
>> Eda
>>
>> Matthew Knepley <knepley at gmail.com>, 1 Tem 2020 Çar, 13:43 tarihinde
>> şunu yazdı:
>> >
>> > On Wed, Jul 1, 2020 at 6:34 AM Eda Oktay <eda.oktay at metu.edu.tr> wrote:
>> >>
>> >> Der Barry,
>> >>
>> >> ı am trying to use your way but I couldn't understand how I create sub
>> >> matrices by using MatCreateSubMatrices() since as input, the function
>> >> needs const IS instead of IS and I couldn't understand how to get a
>> >> const IS. I tried to use ISCreateStride since this IS should be 0:71
>> >> because as you mentioned, the sub matrix should consist of the entire
>> >> matrix. However, since ISCreateStride produces IS, not const IS, I
>> >> couldn't use it in MatCreateSubMatrices().
>> >
>> >
>> > The 'const' refers to the array, not the IS:
>> >
>> >
>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateSubMatrices.html
>> >
>> > Thanks,
>> >
>> > Matt
>> >
>> >>
>> >> Thanks
>> >>
>> >> Eda
>> >>
>> >> Barry Smith <bsmith at petsc.dev>, 10 Haz 2020 Çar, 19:59 tarihinde şunu
>> yazdı:
>> >> >
>> >> >
>> >> > You can use MatCreateSubMatrices() with each process getting a
>> single sequential dense sub matrix that consists of the entire matrix.
>> >> >
>> >> > Use VecDuplicateVecs() to create an array of 72 vectors (create a
>> single seq vector of size 4 as the input to this routine)
>> >> >
>> >> > Then use MatDenseGetArrayRead() to access the upper left corner of
>> the new sequential dense matrix
>> >> >
>> >> > Loop over the vectors calling VecGetArray()
>> >> > Then loop over the row of the dense array filling up the vector
>> >> >
>> >> > Because dense matrices are stored by column, you have to do this
>> looping to fill up the vectors, they can't share the space with the matrix.
>> >> >
>> >> > Barry
>> >> >
>> >> >
>> >> >
>> >> > On Jun 10, 2020, at 11:36 AM, Matthew Knepley <knepley at gmail.com>
>> wrote:
>> >> >
>> >> > On Wed, Jun 10, 2020 at 12:26 PM Eda Oktay <eda.oktay at metu.edu.tr>
>> wrote:
>> >> >>
>> >> >> Matthew Knepley <knepley at gmail.com>, 10 Haz 2020 Çar, 19:13
>> tarihinde
>> >> >> şunu yazdı:
>> >> >> >
>> >> >> > On Wed, Jun 10, 2020 at 12:07 PM Eda Oktay <eda.oktay at metu.edu.tr>
>> wrote:
>> >> >> >>
>> >> >> >> Der Matt,
>> >> >> >>
>> >> >> >> When I looked at the results, I found that there are some
>> problems I
>> >> >> >> couldn't understand.
>> >> >> >>
>> >> >> >> First of all, I am working on a 72*4 matrix and as I said
>> before, I
>> >> >> >> want to have 72 different vectors having size 4 each, whose
>> elements
>> >> >> >> consist of the elements in the same row. And of course, all
>> vectors
>> >> >> >> should be in all processors (currently I am using 4 processors).
>> >> >> >>
>> >> >> >> When I use your scatter code, the output vector is divided into 4
>> >> >> >> parts for 4 processors and each vector consists of 18 row vectors
>> >> >> >> whose elements are arranged in a way that if I want to find
>> zeroth row
>> >> >> >> vector, its elements are located in 0th,18th,36th,54th elements.
>> >> >> >
>> >> >> >
>> >> >> > Was the global size of the vector you wrapped around the dense
>> matrix 72*4?
>> >> >>
>> >> >> Yes it is. I set up its global size to 72*4.
>> >> >>
>> >> >> >
>> >> >> > If you use CreateToAll(), it will make a vector on each process
>> which has the global size of the original vector.
>> >> >>
>> >> >> Although I set 72*4, the size of the vectors in each process is 72.
>> >> >
>> >> >
>> >> > You can understand how it is hard to accept, as this code is tested
>> every night. Can you VecView() the input vector
>> >> > to CreateToAll and the output vector, and send that output?
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Matt
>> >> >
>> >> >>
>> >> >> Thanks,
>> >> >>
>> >> >> Eda
>> >> >>
>> >> >> >
>> >> >> > Thanks,
>> >> >> >
>> >> >> > Matt
>> >> >> >
>> >> >> >>
>> >> >> >> So, isn't scatter's goal is to scatter all values to all
>> processors?
>> >> >> >>
>> >> >> >> Furthermore, I am trying to use my vectors in that way but isn't
>> there
>> >> >> >> any possible way that I can reach my goal entirely?
>> >> >> >>
>> >> >> >> Thanks so much for your help,
>> >> >> >>
>> >> >> >> Eda
>> >> >> >>
>> >> >> >> Matthew Knepley <knepley at gmail.com>, 10 Haz 2020 Çar, 18:11
>> tarihinde
>> >> >> >> şunu yazdı:
>> >> >> >> >
>> >> >> >> > On Wed, Jun 10, 2020 at 10:09 AM Eda Oktay <
>> eda.oktay at metu.edu.tr> wrote:
>> >> >> >> >>
>> >> >> >> >> Dear Matt,
>> >> >> >> >>
>> >> >> >> >> I have one last question I believe. Up to creating a dense
>> matrix I
>> >> >> >> >> did what you've suggested. Thank you so much for that.
>> >> >> >> >>
>> >> >> >> >> I created a new dense matrix. Now, how should I wrap each
>> vector in a
>> >> >> >> >> MatDense again? I mean, what is wrapping vectors in a matrix?
>> To put
>> >> >> >> >> each of them again as rows?
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > I thought you need a dense matrix for something, since you
>> started with one. If you
>> >> >> >> > do not, just do VecGetArray() on the vector from CreateToAll
>> and use the values.
>> >> >> >> >
>> >> >> >> > Thanks,
>> >> >> >> >
>> >> >> >> > Matt
>> >> >> >> >
>> >> >> >> >>
>> >> >> >> >> Thanks!
>> >> >> >> >>
>> >> >> >> >> Eda
>> >> >> >> >>
>> >> >> >> >> Matthew Knepley <knepley at gmail.com>, 10 Haz 2020 Çar, 16:16
>> tarihinde
>> >> >> >> >> şunu yazdı:
>> >> >> >> >> >
>> >> >> >> >> > On Wed, Jun 10, 2020 at 9:08 AM Eda Oktay <
>> eda.oktay at metu.edu.tr> wrote:
>> >> >> >> >> >>
>> >> >> >> >> >> Dear Matt,
>> >> >> >> >> >>
>> >> >> >> >> >> Matthew Knepley <knepley at gmail.com>, 10 Haz 2020 Çar,
>> 16:03 tarihinde
>> >> >> >> >> >> şunu yazdı:
>> >> >> >> >> >> >
>> >> >> >> >> >> > On Wed, Jun 10, 2020 at 8:56 AM Eda Oktay <
>> eda.oktay at metu.edu.tr> wrote:
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> Hi all,
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> I am trying to get all the rows of a parallel matrix as
>> individual
>> >> >> >> >> >> >> vectors. For instance, if I have 72*4 matrix, I want to
>> get 72
>> >> >> >> >> >> >> different vectors having size 4.
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> As far as I understood, MatGetRow is only for local
>> rows, so
>> >> >> >> >> >> >> MatGetOwnershipRange is used, however, when I tried
>> this one, I
>> >> >> >> >> >> >> couldn't get the whole and desired row vectors.
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> In MatGetRow explanation, it is written that I should
>> use
>> >> >> >> >> >> >> MatCreateSubMatrices first, then use MatGetRow. But I
>> couldn't
>> >> >> >> >> >> >> understand to which extent I should create submatrices.
>> I just need to
>> >> >> >> >> >> >> have all 72 rows as 72 different vectors each having 4
>> elements.
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> > 1) For sparse matrices, the storage is always divided by
>> row, so that values can only be retrieved for local rows with MatGetRow()
>> >> >> >> >> >> >
>> >> >> >> >> >> > 2) Is this matrix sparse? It sounds like it is dense.
>> >> >> >> >> >>
>> >> >> >> >> >> Matrix is dense.
>> >> >> >> >> >>
>> >> >> >> >> >> >
>> >> >> >> >> >> > 3) Are you asking to get all matrix values on all
>> processes? If so, I think the easiest thing to do is first wrap a Vec
>> around the
>> >> >> >> >> >> > values, then use VecScatterToAll(), then wrap each
>> one in a MatDense again.
>> >> >> >> >> >>
>> >> >> >> >> >> Yes, I want all row vectors on all processes. In a dense
>> matrix,
>> >> >> >> >> >> should I still wrap a Vec around the values? I know I
>> should use
>> >> >> >> >> >> scatter but I couldn't even wrap a Vec around them.
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > I would do
>> >> >> >> >> >
>> >> >> >> >> > MatGetSize(&N);
>> >> >> >> >> > MatGetLocalSize(&m);
>> >> >> >> >> >
>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatDenseGetArray.html
>> >> >> >> >> > <create vector of local size m*N>
>> >> >> >> >> >
>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecPlaceArray.html
>> >> >> >> >> >
>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatterCreateToAll.html
>> >> >> >> >> > <do scatter>
>> >> >> >> >> >
>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecResetArray.html#VecResetArray
>> >> >> >> >> >
>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateDense.html
>> >> >> >> >> > <use it>
>> >> >> >> >> > <destroy matrix>
>> >> >> >> >> > <destroy vector from CreateToAll>
>> >> >> >> >> >
>> >> >> >> >> > Thanks,
>> >> >> >> >> >
>> >> >> >> >> > Matt
>> >> >> >> >> >
>> >> >> >> >> >>
>> >> >> >> >> >> Thanks so much!
>> >> >> >> >> >>
>> >> >> >> >> >> Eda
>> >> >> >> >> >>
>> >> >> >> >> >> >
>> >> >> >> >> >> > Thanks,
>> >> >> >> >> >> >
>> >> >> >> >> >> > Matt
>> >> >> >> >> >> >
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> Thanks!
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> Eda
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> > --
>> >> >> >> >> >> > What most experimenters take for granted before they
>> begin their experiments is infinitely more interesting than any results to
>> which their experiments lead.
>> >> >> >> >> >> > -- Norbert Wiener
>> >> >> >> >> >> >
>> >> >> >> >> >> > https://www.cse.buffalo.edu/~knepley/
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > --
>> >> >> >> >> > What most experimenters take for granted before they begin
>> their experiments is infinitely more interesting than any results to which
>> their experiments lead.
>> >> >> >> >> > -- Norbert Wiener
>> >> >> >> >> >
>> >> >> >> >> > https://www.cse.buffalo.edu/~knepley/
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > What most experimenters take for granted before they begin
>> their experiments is infinitely more interesting than any results to which
>> their experiments lead.
>> >> >> >> > -- Norbert Wiener
>> >> >> >> >
>> >> >> >> > https://www.cse.buffalo.edu/~knepley/
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> >> >> > -- Norbert Wiener
>> >> >> >
>> >> >> > https://www.cse.buffalo.edu/~knepley/
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> >> > -- Norbert Wiener
>> >> >
>> >> > https://www.cse.buffalo.edu/~knepley/
>> >> >
>> >> >
>> >
>> >
>> >
>> > --
>> > What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> > -- Norbert Wiener
>> >
>> > https://www.cse.buffalo.edu/~knepley/
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200701/7ca00a4e/attachment-0001.html>
More information about the petsc-users
mailing list