<html><body><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><div>OK. I am supposed to destroy the matrix returned by MatISGetMPIXAIJ ?<br></div><div>Also, my example still not get the final assembled local matrix (the MatCreateSubMatrix returns an empty matrix) but as far as I understand my (global) index set is OK: what did I miss ?<br></div><div><br></div><div>Franck<br></div><div><br></div><div><br></div><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>De: </b>"Stefano Zampini" <stefano.zampini@gmail.com><br><b>À: </b>"Franck Houssen" <franck.houssen@inria.fr><br><b>Cc: </b>"petsc-dev" <petsc-dev@mcs.anl.gov>, "PETSc users list" <petsc-users@mcs.anl.gov>, "petsc-maint" <knepley@gmail.com><br><b>Envoyé: </b>Mardi 23 Mai 2017 13:16:18<br><b>Objet: </b>Re: [petsc-dev] Using PETSc MatIS, how to get local matrix (= one domain) before and after assembly ?<br><div><br></div><div dir="auto">MatISGetMPIXAIJ is collective, as it assembles the global operator. To get the matrices you are looking for, you should call MatCreateSubMatrix on the assembled global operator, with the global indices representing the subdomain problem. Each process needs to call both functions<div dir="auto"><br></div><div dir="auto">Stefano</div></div><div class="gmail_extra"><br><div class="gmail_quote">Il 23 Mag 2017 11:41, "Franck Houssen" <<a href="mailto:franck.houssen@inria.fr" target="_blank">franck.houssen@inria.fr</a>> ha scritto:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-family:times new roman,new york,times,serif;font-size:12pt;color:#000000"><div>I have a 3x3 global matrix made of two overlapping 2x2 local matrix (= diagonal with 1.). Each local matrix correspond to one domain (each domain is delegated to one MPI proc, so, I have 2 MPI procs because I have 2 domains).</div><div>This is the simplest possible example: I have two 2x2 (local) diag matrix that overlap so that the global matrix built from them is 1, 2, 1 on the diagonal (local contributions add up in the middle).<br></div><div><br></div><div>Now, I need for each MPI proc to get the assembled local matrix (sometimes called the dirichlet matrix) : this is a local matrix (sequential - not distributed with MPI) that accounts for contribution of neighboring domains (MPI proc).<br></div><div><br></div><div>How to get the local assembled matrix ? MatGetLocalSubMatrix does not work (throw error - see example attached). MatGetSubMatrix returns a MPI distributed matrix, not a local (sequential) one.<br></div><div><ol><li>My understanding is that MatISGetMPIXAIJ should return a local matrix (sequential AIJ matrix) : the MPI in the name recall that you get the assembled matrix (with contributions from the shared border) from the other MPI processus. Correct ? In my simple example, I replaced MatGetLocalSubMatrix with MatISGetMPIXAIJ : I get a deadlock which was surprising to me... Is MatISGetMPIXAIJ a collective call ?<br></li><li>Supposing this is a collective call (and that point 1 is not correct), I ride up MatISGetMPIXAIJ before the "if (rank > 0)" : I don't deadlock now, but it seems I get a global matrix which is not the assembled local matrix I am looking for.<br></li><li>I am supposed to destroy the matrix returned by MatISGetMPIXAIJ ? (I believe yes - not sure as AFAIU wording should associate Destroy methods to Create methods)<br></li></ol></div><div>Franck<br></div><div><br></div><div>The git diff illustrate modifications I tried to add to the initial file attached to this thread:<br></div><div>--- a/matISLocalMat.cpp<br>+++ b/matISLocalMat.cpp<br>@@ -31,6 +31,8 @@ int main(int argc,char **argv) {<br> MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY);<br> MatView(A, PETSC_VIEWER_STDOUT_WORLD); PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD); // Diag: 1, 2, 1<br> <br>+ Mat assembledLocalMat;<br>+ MatISGetMPIXAIJ(A, MAT_INITIAL_MATRIX, &assembledLocalMat);<br> if (rank > 0) { // Do not pollute stdout: print only 1 proc<br> std::cout << std::endl << "non assembled local matrix:" << std::endl << std::endl;<br> Mat nonAssembledLocalMat;<br>@@ -38,11 +40,10 @@ int main(int argc,char **argv) {<br> MatView(nonAssembledLocalMat, PETSC_VIEWER_STDOUT_SELF); // Diag: 1, 1<br> <br> std::cout << std::endl << "assembled local matrix:" << std::endl << std::endl;<br>- Mat assembledLocalMat;<br>- IS is; ISCreateGeneral(PETSC_COMM_SELF, localSize, localIdx, PETSC_COPY_VALUES, &is);<br>- MatGetLocalSubMatrix(A, is, is, &assembledLocalMat); // KO ?!...<br>- MatView(assembledLocalMat, PETSC_VIEWER_STDOUT_SELF); // Would like to get => Diag: 2, 1<br>+ //IS is; ISCreateGeneral(PETSC_COMM_SELF, localSize, localIdx, PETSC_COPY_VALUES, &is);<br>+ //MatGetLocalSubMatrix(A, is, is, &assembledLocalMat); // KO ?!...<br> }<br>+ MatView(assembledLocalMat, PETSC_VIEWER_STDOUT_WORLD); // Would like to get => Diag: 2, 1<br><div><br></div></div><div><br></div><hr id="m_7985792947083527645zwchr"><blockquote style="border-left:2px solid #1010ff;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt"><b>De: </b>"Stefano Zampini" <<a href="mailto:stefano.zampini@gmail.com" target="_blank">stefano.zampini@gmail.com</a>><br><b>À: </b>"petsc-maint" <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br><b>Cc: </b>"petsc-dev" <<a href="mailto:petsc-dev@mcs.anl.gov" target="_blank">petsc-dev@mcs.anl.gov</a>>, "PETSc users list" <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>>, "Franck Houssen" <<a href="mailto:franck.houssen@inria.fr" target="_blank">franck.houssen@inria.fr</a>><br><b>Envoyé: </b>Dimanche 21 Mai 2017 22:51:34<br><b>Objet: </b>Re: [petsc-dev] Using PETSc MatIS, how to get local matrix (= one domain) before and after assembly ?<br><div><br></div><div dir="auto">To assemble the operator in aij format, use <div dir="auto">MatISGetMPIXAIJ</div><div dir="auto"><a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatISGetMPIXAIJ.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatISGetMPIXAIJ.html</a><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">Il 21 Mag 2017 18:43, "Matthew Knepley" <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> ha scritto:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Sun, May 21, 2017 at 11:23 AM, Franck Houssen <span dir="ltr"><<a href="mailto:franck.houssen@inria.fr" target="_blank">franck.houssen@inria.fr</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="font-family:"times new roman","new york",times,serif;font-size:12pt;color:rgb(0,0,0)"><div>I have a 3x3 global matrix is built (diag: 1, 2, 1): it's made of 2 overlapping 2x2 local matrix (diag: 1, 1).</div><div>Getting non assembled local matrix is OK with MatISGetLocalMat.</div><div>How to get assembled local matrix (initial local matrix + neigbhor contributions on the borders) ? (expected result is diag: 2, 1)</div></div></div></blockquote><div><br></div><div>You can always use</div><div><br></div><div><a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetSubMatrix.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetSubMatrix.html</a><br></div><div><a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetSubMatrices.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetSubMatrices.html</a><br></div><div><br></div><div>to get copies, but if you just want to build things, you can use</div><div><br></div><div><a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetLocalSubMatrix.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetLocalSubMatrix.html</a><br></div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="font-family:"times new roman","new york",times,serif;font-size:12pt;color:rgb(0,0,0)"><div>Franck<br></div></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_7985792947083527645m_-6074799036086587145gmail_signature"><div dir="ltr"><div class="m_7985792947083527645m_-6074799036086587145gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div class="m_7985792947083527645m_-6074799036086587145gmail_signature"><br></div><div class="m_7985792947083527645m_-6074799036086587145gmail_signature"><a href="http://www.caam.rice.edu/~mk51/" target="_blank">http://www.caam.rice.edu/~mk51/</a><br></div></div></div></div></div></blockquote></div></div></blockquote><div><br></div></div></div></blockquote></div></div>
</blockquote><div><br></div></div></body></html>