<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hello Yann,<div class="">MatGetColumnVector() is indeed kind of slow and there is a hard copy under the hood.</div><div class="">Better to use MatDenseGetColumnVec()/MatDenseRestoreColumn() which is more efficient (<a href="https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatDenseGetColumn.html" class="">https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatDenseGetColumn.html</a>), or do what Barry suggested.</div><div class=""><br class=""></div><div class="">KSPSetMatSolveBlockSize() is a rather advanced option, it’s only used by KSPHPDDM, not KSPPREONLY (I think?), which I’m guessing is what you are using with a full LU.</div><div class="">The idea behind this option, borrowed from MUMPS option ICNTL(27), is to solve multiple systems with multiple RHS.</div><div class="">So you split AX = B into A[X_0, X_1, …, X_p] = [B_0, B_1, …, B_p] where all blocks are of width at most KSPMatSolveBlockSize.</div><div class="">It’s useful when you have a large number of right-hand sides and it’s becoming too memory-demanding to fit everything in memory, e.g., when doing block Krylov methods or out-of-core LU factorization.</div><div class="">However, if everything fits, it’s best to just do KSPSetMatSolveBlockSize(ksp, PETSC_DECIDE) or don’t call that routine at all, which will default to a single solve with a single thick block.</div><div class="">Direct solvers scale really nicely with the number of right-hand sides, thanks to BLAS3, so the higher the number of RHS, the better the efficiency.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Pierre</div><div class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 17 Sep 2020, at 11:36 PM, Yann Jobic <<a href="mailto:yann.jobic@univ-amu.fr" class="">yann.jobic@univ-amu.fr</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Hi Stefano,<br class="">You're right, i just switched the rows and colomns, and it's working fine. However, i've done this in order to have better performance when i access the rows, with MatGetRow. With this order, i need MatGetColumnVector, which is believe is kind of slow, as stated in the comments of the implementation (but i couldn't find it again, maybe i was looking at an old implementation?).<br class="">I still don't understand why i can not use the transpose of this matrix, as i give the right parallel decomposition. It should be a bijective operation no ?<br class="">I think i'll be using the KSPMatSolve of Pierre, so i don't have to redevelop this part.<br class="">Thanks a lot,<br class="">Yann<br class=""><br class=""><br class="">Le 9/17/2020 à 6:29 PM, Stefano Zampini a écrit :<br class=""><blockquote type="cite" class="">Yann<br class=""> you want to have the number of columns equal to the number of rhs. Not the number of rows.<br class="">This is also consistent in terms of logical layouts. Simply create and dump the matrix this way and you can read in parallel splitting yhe rows ( ie the logical size per process of the vectors)<br class="">Il Gio 17 Set 2020, 18:09 Pierre Jolivet <<a href="mailto:pierre.jolivet@enseeiht.fr" class="">pierre.jolivet@enseeiht.fr</a> <<a href="mailto:pierre.jolivet@enseeiht.fr" class="">mailto:pierre.jolivet@enseeiht.fr</a>>> ha scritto:<br class=""> Hello Yann,<br class=""> This is probably not fully answering your question, but the proper<br class=""> way to solve a system with N RHS is _not_ to use KSPSolve(), but<br class=""> instead KSPMatSolve(), cf.<br class=""> <a href="https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/KSP/KSPMatSolve.html" class="">https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/KSP/KSPMatSolve.html</a>.<br class=""> If you are tracking master (from the GitLab repository), it’s<br class=""> available out of the box. If you are using the release tarballs, it<br class=""> will be available in 3.14.0 scheduled to be released in a couple of<br class=""> days.<br class=""> If you want to know more about the current status of block solvers<br class=""> in PETSc, please feel free to have a look at this preprint:<br class=""> <a href="http://jolivet.perso.enseeiht.fr/article.pdf" class="">http://jolivet.perso.enseeiht.fr/article.pdf</a><br class=""> If you are using a specific PC which is not “multi-RHS ready”, see<br class=""> the list at the top of page 5, please let me know and I’ll tell you<br class=""> how easy to support it.<br class=""> Thanks,<br class=""> Pierre<br class=""></blockquote></div></div></blockquote></div><br class=""></div></body></html>