<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 6 Jun 2017, at 11:21 PM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" class="">bsmith@mcs.anl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><br class="Apple-interchange-newline">On Jun 6, 2017, at 1:52 PM, Pierre Jolivet <<a href="mailto:Pierre.Jolivet@enseeiht.fr" class="">Pierre.Jolivet@enseeiht.fr</a>> wrote:<br class=""><br class="">On Tue, 6 Jun 2017 13:44:06 -0500, Hong wrote:<br class=""><blockquote type="cite" class="">Pierre :<br class=""><br class=""><blockquote type="cite" class="">Yes, of course I defined (*C)->ops->matmultnumeric =<br class="">MatMatMultNumeric_MPIBAIJ_MPIDense in<br class="">MatMatMultSymbolic_MPIBAIJ_MPIDense.<br class="">However, the routine MatMatMultSymbolic_MPIBAIJ_MPIDense is never<br class="">reached when calling MatMatMult with scall == MAT_REUSE_MATRIX<br class=""><br class=""></blockquote>(<a href="http://www.mcs.anl.gov/petsc/petsc-current/src/mat/interface/matrix.c.html#line9487" class="">http://www.mcs.anl.gov/petsc/petsc-current/src/mat/interface/matrix.c.html#line9487</a><br class=""><blockquote type="cite" class="">[1], just to be sure I added a dummy printf in<br class="">MatMatMultSymbolic_MPIBAIJ_MPIDense and nothing is displayed with<br class="">MAT_REUSE_MATRIX, MAT_INITIAL_MATRIX works as intended)<br class=""></blockquote><br class="">MatMatMultSymbolic_xxx() is called only for MAT_INITIAL_MATRIX,<br class="">during which, it defines<br class="">(*C)->ops->matmultnumeric = MatMatMultNumeric_MPIBAIJ_MPIDense;<br class=""></blockquote><br class="">Sorry, I was not thorough enough. What I meant to say is that I never call MatMatMult with MAT_INITIAL_MATRIX, so MatMatMultSymbolic_xxx is never called.<br class=""></blockquote><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""> It is not designed to work that way. The reuse is specifically to reuse a matrix obtained with a call to initial_matrix, not to reuse a random matrix the user provides. The reason is that the symbolic stage may/does build additional data structures that are used for the multiples that the user cannot see or build directly.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></blockquote><div><br class=""></div><div>I’m doing:</div><div><div> MatCreateDense(PETSC_COMM_WORLD, m, n, M, N, in, &B);</div><div><div> MatCreateDense(PETSC_COMM_WORLD, m, n, M, N, out, &C);</div><div> MatMatMult(A, B, MAT_REUSE_MATRIX, PETSC_DEFAULT, &C);</div><div><br class=""></div><div class="">This works perfectly fine with A of type MPIAIJ, but once again, everything is hardwired for this only type.</div><div class=""><br class=""></div></div></div><div>According to your answer, Barry, I think the manual should be adjusted accordingly because to me what I’m trying to achieve should be possible as is (the fact that it is possible with MPIAIJ is also puzzling):</div><div>"In the special case where matrix B (and hence C) are dense you can create the correctly sized matrix C yourself and then call this routine with MAT_REUSE_MATRIX, rather than first having MatMatMult() create it for you. […]” (<a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMatMult.html" class="">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMatMult.html</a>)</div><br class=""><blockquote type="cite" class=""><div class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">Is there another way?<br class="">I cannot use MAT_INITIAL_MATRIX because I want to manage the underlying memory, and I'm guessing MAT_INITIAL_MATRIX will delete whatever is in C and reallocate memory on its own.<br class=""></blockquote><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""> The memory in C is just a dense array so presumably you want to use some "application" memory for this space?<span class="Apple-converted-space"> </span></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""> The simplest way for you to achieve this is to call the MatMatMultSymbolic() then dig out the underly array of the matrix, call PetscFree() on it and put in your own array. Yes, this is slightly hacky, but it will work fine within the current PETSc model.<span class="Apple-converted-space"> </span></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></blockquote><div><br class=""></div><div>Yeah, alright, I already need to tinker with the MatDensePlaceArray/MatDenseResetArray so I can figure something out as well for this.</div><div>Thanks,</div><div>Pierre</div><div><br class=""></div><blockquote type="cite" class=""><div class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""> Barry</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><br class="">Thanks in advance for your help,<br class="">Pierre<br class=""><br class=""><blockquote type="cite" class="">Then MatMatMult(A,C,MAT_REUSE_MATRIX,..) calls<span class="Apple-converted-space"> </span><br class="">(*(*C)->ops->matmultnumeric)(A,B,*C); (line 9432 in matrix.c)<br class=""><br class="">which should go to MatMatMultNumeric_MPIBAIJ_MPIDense.<br class=""><br class="">You may follow a debugging process<br class="">using petsc/src/mat/examples/tests/ex109.c<br class=""><br class="">Are you working on a branch of petsc? If so, I may take a look and<br class="">see what is the problem.<br class=""><br class="">Hong<br class=""><br class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class="">2) I'm having trouble when scall == MAT_REUSE_MATRIX. Here,<br class=""></blockquote><br class=""><br class=""><br class=""></blockquote><br class=""></blockquote><a href="http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/dense/mpi/mpidense.c.html#line1208" class="">http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/dense/mpi/mpidense.c.html#line1208</a><br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">[2]<br class=""><br class=""><blockquote type="cite" class="">[2] it looks that the numeric part of the MatMatMult (which is<br class="">called when scall == MAT_REUSE_MATRIX) is hardwired to this<br class="">routine<br class=""></blockquote><br class=""><br class=""><br class=""></blockquote><br class=""></blockquote>http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/aij/mpi/mpimatmatmult.c.html#line376<br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">[3]<br class="">[3].<span class="Apple-converted-space"> </span><br class="">br><br class=""><br class=""></blockquote><br class=""></blockquote>s.anl.gov/petsc/petsc-current/src/mat/impls/aij/mpi/mpimatmatmult.c.html#line556"<br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">rel="noreferrer"<br class=""><br class=""></blockquote><br class=""></blockquote>target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/aij/mpi/mpimatmatmult.c.html#line556<br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">[2]<br class=""><br class=""><br class=""><br class=""></blockquote><br class=""></blockquote>http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/dense/mpi/mpidense.c.html#line1208<br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">[4]<br class="">[3]<br class=""><br class=""><br class=""><br class=""></blockquote><br class=""></blockquote>http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/aij/mpi/mpimatmatmult.c.html#line376<br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">[5]<br class=""></blockquote></blockquote><br class=""><br class=""><br class="">Links:<br class="">------<br class="">[1]<br class="">http://www.mcs.anl.gov/petsc/petsc-current/src/mat/interface/matrix.c.html#line9487<br class="">[2]<br class="">http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/dense/mpi/mpidense.c.html#line1208<br class="">[3]<br class="">http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/aij/mpi/mpimatmatmult.c.html#line376<br class="">[4]<br class="">http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/dense/mpi/mpidense.c.html#line1208<br class="">[5]<br class="">http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/aij/mpi/mpimatmatmult.c.html#line376</blockquote></blockquote></div></blockquote></div><br class=""></body></html>