[petsc-users] MatGetRow for global rows of a parallel matrix

Barry Smith bsmith at petsc.dev
Wed Jun 10 11:59:12 CDT 2020 

  You can use MatCreateSubMatrices() with each process getting a single sequential dense sub matrix that consists of the entire matrix.

  Use VecDuplicateVecs() to create an array of 72 vectors (create a single seq vector of size 4 as the input to this routine)

  Then use MatDenseGetArrayRead() to access the upper left corner of the new sequential dense matrix

  Loop over the vectors calling VecGetArray()
     Then loop over the row of the dense array filling up the vector 

  Because dense matrices are stored by column, you have to do this looping to fill up the vectors, they can't share the space with the matrix.

  Barry



> On Jun 10, 2020, at 11:36 AM, Matthew Knepley <knepley at gmail.com> wrote:
> 
> On Wed, Jun 10, 2020 at 12:26 PM Eda Oktay <eda.oktay at metu.edu.tr <mailto:eda.oktay at metu.edu.tr>> wrote:
> Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>>, 10 Haz 2020 Çar, 19:13 tarihinde
> şunu yazdı:
> >
> > On Wed, Jun 10, 2020 at 12:07 PM Eda Oktay <eda.oktay at metu.edu.tr <mailto:eda.oktay at metu.edu.tr>> wrote:
> >>
> >> Der Matt,
> >>
> >> When I looked at the results, I found that there are some problems I
> >> couldn't understand.
> >>
> >> First of all, I am working on a 72*4 matrix and as I said before, I
> >> want to have 72 different vectors having size 4 each, whose elements
> >> consist of the elements in the same row. And of course, all vectors
> >> should be in all processors (currently I am using 4 processors).
> >>
> >> When I use your scatter code, the output vector is divided into 4
> >> parts for 4 processors and each vector consists of 18 row vectors
> >> whose elements are arranged in a way that if I want to find zeroth row
> >> vector, its elements are located in 0th,18th,36th,54th elements.
> >
> >
> > Was the global size of the vector you wrapped around the dense matrix 72*4?
> 
> Yes it is. I set up its global size  to 72*4.
> 
> >
> > If you use CreateToAll(), it will make a vector on each process which has the global size of the original vector.
> 
> Although I set 72*4, the size of the vectors in each process is 72.
> 
> You can understand how it is hard to accept, as this code is tested every night. Can you VecView() the input vector
> to CreateToAll and the output vector,  and send that output?
> 
>   Thanks,
> 
>      Matt
>  
> Thanks,
> 
> Eda
> 
> >
> >   Thanks,
> >
> >     Matt
> >
> >>
> >> So, isn't scatter's goal is to scatter all values to all processors?
> >>
> >> Furthermore, I am trying to use my vectors in that way but isn't there
> >> any possible way that I can reach my goal entirely?
> >>
> >> Thanks so much for your help,
> >>
> >> Eda
> >>
> >> Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>>, 10 Haz 2020 Çar, 18:11 tarihinde
> >> şunu yazdı:
> >> >
> >> > On Wed, Jun 10, 2020 at 10:09 AM Eda Oktay <eda.oktay at metu.edu.tr <mailto:eda.oktay at metu.edu.tr>> wrote:
> >> >>
> >> >> Dear Matt,
> >> >>
> >> >> I have one last question I believe. Up to creating a dense matrix I
> >> >> did what you've suggested. Thank you so much for that.
> >> >>
> >> >> I created a new dense matrix. Now, how should I wrap each vector in a
> >> >> MatDense again? I mean, what is wrapping vectors in a matrix? To put
> >> >> each of them again as rows?
> >> >
> >> >
> >> > I thought you need a dense matrix for something, since you started with one. If you
> >> > do not, just do VecGetArray() on the vector from CreateToAll and use the values.
> >> >
> >> >   Thanks,
> >> >
> >> >      Matt
> >> >
> >> >>
> >> >> Thanks!
> >> >>
> >> >> Eda
> >> >>
> >> >> Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>>, 10 Haz 2020 Çar, 16:16 tarihinde
> >> >> şunu yazdı:
> >> >> >
> >> >> > On Wed, Jun 10, 2020 at 9:08 AM Eda Oktay <eda.oktay at metu.edu.tr <mailto:eda.oktay at metu.edu.tr>> wrote:
> >> >> >>
> >> >> >> Dear Matt,
> >> >> >>
> >> >> >> Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>>, 10 Haz 2020 Çar, 16:03 tarihinde
> >> >> >> şunu yazdı:
> >> >> >> >
> >> >> >> > On Wed, Jun 10, 2020 at 8:56 AM Eda Oktay <eda.oktay at metu.edu.tr <mailto:eda.oktay at metu.edu.tr>> wrote:
> >> >> >> >>
> >> >> >> >> Hi all,
> >> >> >> >>
> >> >> >> >> I am trying to get all the rows of a parallel matrix as individual
> >> >> >> >> vectors. For instance, if I have 72*4 matrix, I want to get 72
> >> >> >> >> different vectors having size 4.
> >> >> >> >>
> >> >> >> >> As far as I understood, MatGetRow is only for local rows, so
> >> >> >> >> MatGetOwnershipRange is used, however, when I tried this one, I
> >> >> >> >> couldn't get the whole and desired row vectors.
> >> >> >> >>
> >> >> >> >> In MatGetRow explanation, it is written that I should use
> >> >> >> >> MatCreateSubMatrices first, then use MatGetRow. But I couldn't
> >> >> >> >> understand to which extent I should create submatrices. I just need to
> >> >> >> >> have all 72 rows as 72 different vectors each having 4 elements.
> >> >> >> >
> >> >> >> >
> >> >> >> > 1) For sparse matrices, the storage is always divided by row, so that values can only be retrieved for local rows with MatGetRow()
> >> >> >> >
> >> >> >> > 2) Is this matrix sparse? It sounds like it is dense.
> >> >> >>
> >> >> >> Matrix is dense.
> >> >> >>
> >> >> >> >
> >> >> >> > 3) Are you asking to get all matrix values on all processes? If so, I think the easiest thing to do is first wrap a Vec around the
> >> >> >> >     values, then use VecScatterToAll(), then wrap each one in a MatDense again.
> >> >> >>
> >> >> >> Yes, I want all row vectors on all processes. In a dense matrix,
> >> >> >> should I still wrap a Vec around the values? I know I should use
> >> >> >> scatter but I couldn't even wrap a Vec around them.
> >> >> >
> >> >> >
> >> >> > I would do
> >> >> >
> >> >> >   MatGetSize(&N);
> >> >> >   MatGetLocalSize(&m);
> >> >> >   https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatDenseGetArray.html <https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatDenseGetArray.html>
> >> >> >   <create vector of local size m*N>
> >> >> >   https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecPlaceArray.html <https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecPlaceArray.html>
> >> >> >   https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatterCreateToAll.html <https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatterCreateToAll.html>
> >> >> >   <do scatter>
> >> >> >   https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecResetArray.html#VecResetArray <https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecResetArray.html#VecResetArray>
> >> >> >   https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateDense.html <https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateDense.html>
> >> >> >   <use it>
> >> >> >   <destroy matrix>
> >> >> >   <destroy vector from CreateToAll>
> >> >> >
> >> >> >   Thanks,
> >> >> >
> >> >> >      Matt
> >> >> >
> >> >> >>
> >> >> >> Thanks so much!
> >> >> >>
> >> >> >> Eda
> >> >> >>
> >> >> >> >
> >> >> >> >   Thanks,
> >> >> >> >
> >> >> >> >      Matt
> >> >> >> >
> >> >> >> >>
> >> >> >> >> Thanks!
> >> >> >> >>
> >> >> >> >> Eda
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> >> >> >> > -- Norbert Wiener
> >> >> >> >
> >> >> >> > https://www.cse.buffalo.edu/~knepley/ <https://www.cse.buffalo.edu/~knepley/>
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> >> >> > -- Norbert Wiener
> >> >> >
> >> >> > https://www.cse.buffalo.edu/~knepley/ <https://www.cse.buffalo.edu/~knepley/>
> >> >
> >> >
> >> >
> >> > --
> >> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> >> > -- Norbert Wiener
> >> >
> >> > https://www.cse.buffalo.edu/~knepley/ <https://www.cse.buffalo.edu/~knepley/>
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> > -- Norbert Wiener
> >
> > https://www.cse.buffalo.edu/~knepley/ <https://www.cse.buffalo.edu/~knepley/>
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
> 
> https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200610/bd5271d0/attachment.html>

More information about the petsc-users mailing list