<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, Aug 4, 2015 at 12:08 PM, Cong Li <span dir="ltr"><<a href="mailto:solvercorleone@gmail.com" target="_blank">solvercorleone@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thanks very much for your suggestions. <div><br></div><div>Actually I am also considering using communication-avoiding matrix power kernel (CA-MPK) to do SPMM. However, the communication pattern of CA-MPK depends on the sparsity pattern. So the implementation could be very complex for some of problems.The efficient implementation of CA-MPK is actually one of problems I want to solve during my PhD course.</div><div><br></div><div>As to the optimisation 2 you suggested, is it the same idea as what Barry Smith suggested?</div><div>I am sorry that I am a Rookie to PETSc, so I am not quite familiar with PETSc implementation strategies.</div></div></blockquote><div><br></div><div>Yes, that is what Barry is suggesting.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Thanks </div><div><br></div><div>Cong Li</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 4, 2015 at 8:34 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Tue, Aug 4, 2015 at 5:31 AM, Cong Li <span dir="ltr"><<a href="mailto:solvercorleone@gmail.com" target="_blank">solvercorleone@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Actually, I am trying to implement s-step krylov subspace method.<div>I want to extend the Krylov subspace by s dimensions by using monomial, which can be defined as C={X, AX, A^2X, ... , A^sX}, in one loop. So, my plan now is to firstly calculate the recurrence, which is P_n(x)=xP_n-1(x), and then use the results to update the items in C. And then, in the next loop of Krylov subspace method, the C will be updated again. This means I need to update C in every iteration.</div><div>This continues till the convergence criteria is satisfied.</div><div><br></div><div>I suppose A is huge sparse SPD matrix with millions of rows, and X is tall-skinny dense matrix.</div><div><br></div><div>Do you still think <span style="font-size:13px">MATNEST is a good way to define C. </span></div><div><span style="font-size:13px"><br></span></div><div>Actually I am wondering if there is a way to do SPMM by using a submatrix of C and also store the result in a submatrix of C. If it is possible, I think we can remove some of cost of data movement.</div><div>For example, C=[c_1, c_2,.., c_s], and I want to use the result of A*c_1 to update c_2, and then use he result of A*c_2(updated) to update c_3 and so on. </div><div>I don't need the intermediate result separately, such as the result of A*c_1, A*c_2. And I only need the final C.</div><div>Is there any SPMM function or strategies I can use to achievement this?</div></div></blockquote><div><br></div></span><div>So there are two optimizations here:</div><div><br></div><div> 1) Communication: You only communicate every s steps. If you are solving a transport dominated problem, this can make sense.</div><div> For elliptic problems, I think it makes no difference at all.</div><div><br></div><div> 2) Computation: You can alleviate bandwidth pressure by acting on multiple vectors at once.</div><div><br></div><div>I would first implement this naively with a collection of Vecs to check that 1) makes a difference for your problem.</div><div>If it does, then I think 2) can best be accomplished by using a TAIJ matrix and a long Vec, where you shift the</div><div>memory at each iterate.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div><div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Thanks</div><div><br></div><div>Cong Li</div><div><div><br><div><span><br></span></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 4, 2015 at 6:46 PM, Patrick Sanan <span dir="ltr"><<a href="mailto:patrick.sanan@gmail.com" target="_blank">patrick.sanan@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>On Tue, Aug 04, 2015 at 06:09:30PM +0900, Cong Li wrote:<br>
> I am sorry that I should have explained it more clearly.<br>
> Actually I want to compute a recurrence.<br>
><br>
> Like, I want to firstly compute A*X1=B1, and then calculate A*B1=B2,<br>
> A*B2=B3 and so on.<br>
> Finally I want to combine all these results into a bigger matrix C=[B1,B2<br>
> ...]<br>
><br>
> Is there any way to do this efficiently.<br>
</span>With no other information about your problem, one literal solution might be to use MATNEST to define C once you have computed B1,B2,..<br>
However, this invites questions about what you plan to do with C and whether you require explicit representations of some or all of these matrices, and what problem sizes you are considering.<br>
<div><div>><br>
><br>
><br>
> On Tue, Aug 4, 2015 at 5:45 PM, Patrick Sanan <<a href="mailto:patrick.sanan@gmail.com" target="_blank">patrick.sanan@gmail.com</a>><br>
> wrote:<br>
><br>
> > On Tue, Aug 04, 2015 at 03:42:14PM +0900, Cong Li wrote:<br>
> > > Thanks for your reply.<br>
> > ><br>
> > > I have an other question.<br>
> > > I want to do SPMM several times and combine result matrices into one<br>
> > bigger<br>
> > > matrix.<br>
> > > for example<br>
> > > I firstly calculate AX1=B1, AX2=B2 ...<br>
> > > then I want to combine B1, B2.. to get a C, where C=[B1,B2...]<br>
> > ><br>
> > > Could you please suggest a way of how to do this.<br>
> > This is just linear algebra, nothing to do with PETSc specifically.<br>
> > A * [X1, X2, ... ] = [AX1, AX2, ...]<br>
> > ><br>
> > > Thanks<br>
> > ><br>
> > > Cong Li<br>
> > ><br>
> > > On Tue, Aug 4, 2015 at 3:27 PM, Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:<br>
> > ><br>
> > > > Cong Li <<a href="mailto:solvercorleone@gmail.com" target="_blank">solvercorleone@gmail.com</a>> writes:<br>
> > > ><br>
> > > > > Hello,<br>
> > > > ><br>
> > > > > I am a PhD student using PETsc for my research.<br>
> > > > > I am wondering if there is a way to implement SPMM (Sparse<br>
> > matrix-matrix<br>
> > > > > multiplication) by using PETSc.<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMatMult.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMatMult.html</a><br>
> > > ><br>
> ><br>
</div></div></blockquote></div><br></div>
</blockquote></div></div></div><br><br clear="all"><span class="HOEnZb"><font color="#888888"><span><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</span></font></span></div></div>
</blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>