<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:12pt"><div><span>Resuming: I have a parabolic differential equation, through the finite difference scheme I obtain a linear system of equations with laplace matrix as coefficient matrix, i.e. the coefficient matrix is sparse, huge and structured. </span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>Then I solve this system with GMRES+Jacob.</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>I don't care about the number of processes and the size of the portions of the matrix
assigned to the processes; of course the matrix is partitioned in blocks assigned to the processes</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>the problem is that I need to know, just theoretically, how is the matrix splitted between processes </span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>Moreover, how does it happen on GPUs??</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>Essentially in which way the domain is splitted between processes? when do they
communicate to synchronize/share/exchange partial results?</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>I nees to know also how does it all happen on a GPU.</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span><br></span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>thank you</span></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style:
normal;"><span><br></span></div><div class="yahoo_quoted" style="display: block;"> <br> <br> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 12pt;"> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 12pt;"> <div dir="ltr"> <font size="2" face="Arial"> Il Mercoledì 8 Gennaio 2014 13:46, Dave May <dave.mayhem23@gmail.com> ha scritto:<br> </font> </div> <div class="y_msg_container"><div id="yiv1802876052"><div>You asked how the problem was split between processes. In your case, this is defined by the matrix.<div><br clear="none"></div><div> The default solver in petsc is gmres preconditioned with block Jacobi and ilu(0) applied on each block. The "block" I refer to is the piece of the matrix locally owned by each processor (which is thus defined by the matrix layout/partitioning)</div>
<div><br clear="none"></div><div><br clear="none"><div><br clear="none"></div><div class="yiv1802876052yqt2904431728" id="yiv1802876052yqt80801"><div><br clear="none"><br clear="none">On Wednesday, 8 January 2014, mary sweat wrote:<br clear="none"><blockquote class="yiv1802876052gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div style="font-size: 12pt; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;">
<div>I do not explicitily check the size, because I use <span style="font-family: 'Helvetica Neue', 'Segoe UI', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 13px;">PETSC_DECIDE, instead I specify the number of processes. What I really care about is hw does KSPSolve solve the system in a parallel way with multiple processes.</span></div>
<div style="display:block;"> <br clear="none"> <br clear="none"> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 12pt;"> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 12pt;">
<div dir="ltr"> <font face="Arial"> Il Mercoledì 8 Gennaio 2014 12:34, Dave May <<a rel="nofollow" shape="rect" href="">dave.mayhem23@gmail.com</a>> ha scritto:<br clear="none">
</font> </div> <div><div><div><div dir="ltr"><div><div><div><div><div><div>Please check out the manual page for MatSetSizes()<br clear="none"> <a rel="nofollow" shape="rect" target="_blank" href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetSizes.html">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetSizes.html</a><br clear="none">
<br clear="none"></div>Basically you have two choices:<br clear="none"></div><br clear="none">1/ Define the global size of the matrix and use PETSC_DECIDE for the local sizes.<br clear="none"></div>In this case, PETSc will define the local row size in a manner such that there are approximately the same number of rows on each process.<br clear="none">
</div><br clear="none">2/ Define the local sizes yourself and use PETSC_DETERMINE for the global size. <br clear="none">Then you have full control over the parallel layout.<br clear="none"><br clear="none"></div>The following functions described by these pages<br clear="none">
<br clear="none"> <a rel="nofollow" shape="rect" target="_blank" href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetSize.html">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetSize.html</a><br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetLocalSize.html">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetLocalSize.html</a><br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetOwnershipRanges.html">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetOwnershipRanges.html</a><br clear="none">
</div> <a rel="nofollow" shape="rect" target="_blank" href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetOwnershipRangesColumn.html">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetOwnershipRangesColumn.html</a><br clear="none">
<br clear="none">
might also be useful for you in double checking what the matrix decomposition looks like<br clear="none"><div><div><br clear="none"><div><div><div><div><br clear="none"></div><div>Cheers,<br clear="none"></div><div> Dave<br clear="none">
</div><div><br clear="none"></div></div></div></div></div></div>
</div><div><div><br clear="none"><br clear="none"><div>On 8 January 2014 12:26, mary sweat <span dir="ltr"><<a rel="nofollow" shape="rect" href="">mary.sweat78@yahoo.it</a>></span> wrote:<br clear="none">
<blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div style="font-size: 12pt; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;"><div>
My target is the following. I got a huge linear system with a sparse huge matrix, nothing to deal with PDE. How is the system splitted between processes? is there in this suggested book the answer?</div><div>Thanks again</div>
<div style="display:block;"> <br clear="none"> <br clear="none"> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 12pt;"> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 12pt;">
<div dir="ltr"> <font face="Arial"> Il Martedì 7 Gennaio 2014 17:34, Jed Brown <<a rel="nofollow" shape="rect" href="">jedbrown@mcs.anl.gov</a>> ha scritto:<br clear="none">
</font> </div><div><div> <div>
mary sweat <<a rel="nofollow" shape="rect" href="">mary.sweat78@yahoo.it</a>> writes:<div><br clear="none"><br clear="none">> Hi all, I need to know how does KSP separate and distribute domain<br clear="none">
> between processes and the way processes share and communicate halfway<br clear="none">> results. Is there any good documentation about it???</div><br clear="none"><br clear="none">The communication is in Mat and Vec functions. You can see it<br clear="none">
summarized in -log_summary. For the underlying theory, see Barry's<br clear="none">book.<br clear="none"><br clear="none"><a rel="nofollow" shape="rect" target="_blank" href="http://www.mcs.anl.gov/~bsmith/ddbook.html">http://www.mcs.anl.gov/~bsmith/ddbook.html</a><br clear="none">
<br clear="none"></div> </div></div></div> </div> </div> </div></div></blockquote></div><br clear="none"></div></div></div></div><br clear="none"><br clear="none"></div> </div> </div> </div> </div></div></blockquote></div></div></div></div></div><br><br></div> </div> </div> </div> </div></body></html>