<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> You can use MatCreateSubMatrices() with each process getting a single sequential dense sub matrix that consists of the entire matrix.<div class=""><br class=""></div><div class=""> Use VecDuplicateVecs() to create an array of 72 vectors (create a single seq vector of size 4 as the input to this routine)</div><div class=""><br class=""></div><div class=""> Then use MatDenseGetArrayRead() to access the upper left corner of the new sequential dense matrix</div><div class=""><br class=""></div><div class=""> Loop over the vectors calling VecGetArray()</div><div class=""> Then loop over the row of the dense array filling up the vector </div><div class=""><br class=""></div><div class=""> Because dense matrices are stored by column, you have to do this looping to fill up the vectors, they can't share the space with the matrix.</div><div class=""><br class=""></div><div class=""> Barry</div><div class=""><br class=""></div><div class=""><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jun 10, 2020, at 11:36 AM, Matthew Knepley <<a href="mailto:knepley@gmail.com" class="">knepley@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class="">On Wed, Jun 10, 2020 at 12:26 PM Eda Oktay <<a href="mailto:eda.oktay@metu.edu.tr" class="">eda.oktay@metu.edu.tr</a>> wrote:<br class=""></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>>, 10 Haz 2020 Çar, 19:13 tarihinde<br class="">
şunu yazdı:<br class="">
><br class="">
> On Wed, Jun 10, 2020 at 12:07 PM Eda Oktay <<a href="mailto:eda.oktay@metu.edu.tr" target="_blank" class="">eda.oktay@metu.edu.tr</a>> wrote:<br class="">
>><br class="">
>> Der Matt,<br class="">
>><br class="">
>> When I looked at the results, I found that there are some problems I<br class="">
>> couldn't understand.<br class="">
>><br class="">
>> First of all, I am working on a 72*4 matrix and as I said before, I<br class="">
>> want to have 72 different vectors having size 4 each, whose elements<br class="">
>> consist of the elements in the same row. And of course, all vectors<br class="">
>> should be in all processors (currently I am using 4 processors).<br class="">
>><br class="">
>> When I use your scatter code, the output vector is divided into 4<br class="">
>> parts for 4 processors and each vector consists of 18 row vectors<br class="">
>> whose elements are arranged in a way that if I want to find zeroth row<br class="">
>> vector, its elements are located in 0th,18th,36th,54th elements.<br class="">
><br class="">
><br class="">
> Was the global size of the vector you wrapped around the dense matrix 72*4?<br class="">
<br class="">
Yes it is. I set up its global size to 72*4.<br class="">
<br class="">
><br class="">
> If you use CreateToAll(), it will make a vector on each process which has the global size of the original vector.<br class="">
<br class="">
Although I set 72*4, the size of the vectors in each process is 72.<br class=""></blockquote><div class=""><br class=""></div><div class="">You can understand how it is hard to accept, as this code is tested every night. Can you VecView() the input vector</div><div class="">to CreateToAll and the output vector, and send that output?</div><div class=""><br class=""></div><div class=""> Thanks,</div><div class=""><br class=""></div><div class=""> Matt</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Thanks,<br class="">
<br class="">
Eda<br class="">
<br class="">
><br class="">
> Thanks,<br class="">
><br class="">
> Matt<br class="">
><br class="">
>><br class="">
>> So, isn't scatter's goal is to scatter all values to all processors?<br class="">
>><br class="">
>> Furthermore, I am trying to use my vectors in that way but isn't there<br class="">
>> any possible way that I can reach my goal entirely?<br class="">
>><br class="">
>> Thanks so much for your help,<br class="">
>><br class="">
>> Eda<br class="">
>><br class="">
>> Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>>, 10 Haz 2020 Çar, 18:11 tarihinde<br class="">
>> şunu yazdı:<br class="">
>> ><br class="">
>> > On Wed, Jun 10, 2020 at 10:09 AM Eda Oktay <<a href="mailto:eda.oktay@metu.edu.tr" target="_blank" class="">eda.oktay@metu.edu.tr</a>> wrote:<br class="">
>> >><br class="">
>> >> Dear Matt,<br class="">
>> >><br class="">
>> >> I have one last question I believe. Up to creating a dense matrix I<br class="">
>> >> did what you've suggested. Thank you so much for that.<br class="">
>> >><br class="">
>> >> I created a new dense matrix. Now, how should I wrap each vector in a<br class="">
>> >> MatDense again? I mean, what is wrapping vectors in a matrix? To put<br class="">
>> >> each of them again as rows?<br class="">
>> ><br class="">
>> ><br class="">
>> > I thought you need a dense matrix for something, since you started with one. If you<br class="">
>> > do not, just do VecGetArray() on the vector from CreateToAll and use the values.<br class="">
>> ><br class="">
>> > Thanks,<br class="">
>> ><br class="">
>> > Matt<br class="">
>> ><br class="">
>> >><br class="">
>> >> Thanks!<br class="">
>> >><br class="">
>> >> Eda<br class="">
>> >><br class="">
>> >> Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>>, 10 Haz 2020 Çar, 16:16 tarihinde<br class="">
>> >> şunu yazdı:<br class="">
>> >> ><br class="">
>> >> > On Wed, Jun 10, 2020 at 9:08 AM Eda Oktay <<a href="mailto:eda.oktay@metu.edu.tr" target="_blank" class="">eda.oktay@metu.edu.tr</a>> wrote:<br class="">
>> >> >><br class="">
>> >> >> Dear Matt,<br class="">
>> >> >><br class="">
>> >> >> Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>>, 10 Haz 2020 Çar, 16:03 tarihinde<br class="">
>> >> >> şunu yazdı:<br class="">
>> >> >> ><br class="">
>> >> >> > On Wed, Jun 10, 2020 at 8:56 AM Eda Oktay <<a href="mailto:eda.oktay@metu.edu.tr" target="_blank" class="">eda.oktay@metu.edu.tr</a>> wrote:<br class="">
>> >> >> >><br class="">
>> >> >> >> Hi all,<br class="">
>> >> >> >><br class="">
>> >> >> >> I am trying to get all the rows of a parallel matrix as individual<br class="">
>> >> >> >> vectors. For instance, if I have 72*4 matrix, I want to get 72<br class="">
>> >> >> >> different vectors having size 4.<br class="">
>> >> >> >><br class="">
>> >> >> >> As far as I understood, MatGetRow is only for local rows, so<br class="">
>> >> >> >> MatGetOwnershipRange is used, however, when I tried this one, I<br class="">
>> >> >> >> couldn't get the whole and desired row vectors.<br class="">
>> >> >> >><br class="">
>> >> >> >> In MatGetRow explanation, it is written that I should use<br class="">
>> >> >> >> MatCreateSubMatrices first, then use MatGetRow. But I couldn't<br class="">
>> >> >> >> understand to which extent I should create submatrices. I just need to<br class="">
>> >> >> >> have all 72 rows as 72 different vectors each having 4 elements.<br class="">
>> >> >> ><br class="">
>> >> >> ><br class="">
>> >> >> > 1) For sparse matrices, the storage is always divided by row, so that values can only be retrieved for local rows with MatGetRow()<br class="">
>> >> >> ><br class="">
>> >> >> > 2) Is this matrix sparse? It sounds like it is dense.<br class="">
>> >> >><br class="">
>> >> >> Matrix is dense.<br class="">
>> >> >><br class="">
>> >> >> ><br class="">
>> >> >> > 3) Are you asking to get all matrix values on all processes? If so, I think the easiest thing to do is first wrap a Vec around the<br class="">
>> >> >> > values, then use VecScatterToAll(), then wrap each one in a MatDense again.<br class="">
>> >> >><br class="">
>> >> >> Yes, I want all row vectors on all processes. In a dense matrix,<br class="">
>> >> >> should I still wrap a Vec around the values? I know I should use<br class="">
>> >> >> scatter but I couldn't even wrap a Vec around them.<br class="">
>> >> ><br class="">
>> >> ><br class="">
>> >> > I would do<br class="">
>> >> ><br class="">
>> >> > MatGetSize(&N);<br class="">
>> >> > MatGetLocalSize(&m);<br class="">
>> >> > <a href="https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatDenseGetArray.html" rel="noreferrer" target="_blank" class="">https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatDenseGetArray.html</a><br class="">
>> >> > <create vector of local size m*N><br class="">
>> >> > <a href="https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecPlaceArray.html" rel="noreferrer" target="_blank" class="">https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecPlaceArray.html</a><br class="">
>> >> > <a href="https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatterCreateToAll.html" rel="noreferrer" target="_blank" class="">https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatterCreateToAll.html</a><br class="">
>> >> > <do scatter><br class="">
>> >> > <a href="https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecResetArray.html#VecResetArray" rel="noreferrer" target="_blank" class="">https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecResetArray.html#VecResetArray</a><br class="">
>> >> > <a href="https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateDense.html" rel="noreferrer" target="_blank" class="">https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateDense.html</a><br class="">
>> >> > <use it><br class="">
>> >> > <destroy matrix><br class="">
>> >> > <destroy vector from CreateToAll><br class="">
>> >> ><br class="">
>> >> > Thanks,<br class="">
>> >> ><br class="">
>> >> > Matt<br class="">
>> >> ><br class="">
>> >> >><br class="">
>> >> >> Thanks so much!<br class="">
>> >> >><br class="">
>> >> >> Eda<br class="">
>> >> >><br class="">
>> >> >> ><br class="">
>> >> >> > Thanks,<br class="">
>> >> >> ><br class="">
>> >> >> > Matt<br class="">
>> >> >> ><br class="">
>> >> >> >><br class="">
>> >> >> >> Thanks!<br class="">
>> >> >> >><br class="">
>> >> >> >> Eda<br class="">
>> >> >> ><br class="">
>> >> >> ><br class="">
>> >> >> ><br class="">
>> >> >> > --<br class="">
>> >> >> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">
>> >> >> > -- Norbert Wiener<br class="">
>> >> >> ><br class="">
>> >> >> > <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class="">
>> >> ><br class="">
>> >> ><br class="">
>> >> ><br class="">
>> >> > --<br class="">
>> >> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">
>> >> > -- Norbert Wiener<br class="">
>> >> ><br class="">
>> >> > <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class="">
>> ><br class="">
>> ><br class="">
>> ><br class="">
>> > --<br class="">
>> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">
>> > -- Norbert Wiener<br class="">
>> ><br class="">
>> > <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class="">
><br class="">
><br class="">
><br class="">
> --<br class="">
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">
> -- Norbert Wiener<br class="">
><br class="">
> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class="">
</blockquote></div><br clear="all" class=""><div class=""><br class=""></div>-- <br class=""><div dir="ltr" class="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class=""></div></div></div></div></div></div></div></div>
</div></blockquote></div><br class=""></div></div></body></html>