<div dir="ltr"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Pierre:</span><div>I added this support </div><div><a href="https://bitbucket.org/petsc/petsc/commits/ef7efd37182b12f7b7ab7c34390aae8e592a83c2">https://bitbucket.org/petsc/petsc/commits/ef7efd37182b12f7b7ab7c34390aae8e592a83c2</a></div><div><br></div><div>It is in branch hzhang/fix-pc-fieldsplit-reuse.</div><div>You may give it a try. Once it passed all our regression tests, it will be merged to petsc-master branch.</div><div><br></div><div>Your modified ex42.c is very helpful, thanks!</div><div>Hong<br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 4, 2018 at 10:15 AM, Hong <span dir="ltr"><<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Pierre:</div><div class="gmail_quote">Thanks for creating this use case. I'll check it and get back to you sometime next week.</div><div class="gmail_quote"><span class="gmail-HOEnZb"><font color="#888888">Hong</font></span><div><div class="gmail-h5"><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word">Hong,<div>Thanks for your reply.</div><div>It’s not so minimalist, but at least it is based on one of your example.</div><div><div>diff --git a/src/ksp/ksp/examples/tutoria<wbr>ls/ex42.c b/src/ksp/ksp/examples/tutoria<wbr>ls/ex42.c</div><div>index 8be4389a2d..520c82569a 100644</div><div>--- a/src/ksp/ksp/examples/tutoria<wbr>ls/ex42.c</div><div>+++ b/src/ksp/ksp/examples/tutoria<wbr>ls/ex42.c</div><div>@@ -1953,6 +1953,15 @@ static PetscErrorCode solve_stokes_3d_coupled(PetscI<wbr>nt mx,PetscInt my,PetscInt m</div><div>     }</div><div>   }</div><div>   ierr = KSPSolve(ksp_S,f,X);CHKERRQ(ie<wbr>rr);</div><div>+  PetscViewerPushFormat(PETSC_V<wbr>IEWER_STDOUT_WORLD, PETSC_VIEWER_ASCII_INFO);</div><div>+  MatView(A, PETSC_VIEWER_STDOUT_WORLD);</div><div>+  Mat C;</div><div>+  MatMatMult(A, A, MAT_INITIAL_MATRIX, PETSC_DEFAULT, &C);</div><div>+  MatView(C, PETSC_VIEWER_STDOUT_WORLD);</div><div>+  PetscViewerPopFormat(PETSC_VI<wbr>EWER_STDOUT_WORLD);</div><div>+  ierr = KSPSetOperators(ksp_S,C,C);CHK<wbr>ERRQ(ierr);</div><div>+  ierr = KSPSolve(ksp_S,f,X);CHKERRQ(ie<wbr>rr);</div><div>+  MatDestroy(&C);</div><div><br></div><div>   ierr = PetscOptionsGetBool(NULL,NULL,<wbr>"-write_pvts",&write_output,NU<wbr>LL);CHKERRQ(ierr);</div><div>   if (write_output) {ierr = DAView3DPVTS(da_Stokes,X,"up")<wbr>;CHKERRQ(ierr);}</div><div><br></div><div>$ mpirun -np 1 ./ex42 -mx 2 -my 2 -mz 2 -stokes_pc_type fieldsplit</div><div>This should give you this exact error.</div><div>Do you know of a way to bypass the error by flushing/destroying the PC while still keeping the options for the following KSPSolve?</div><div><br></div><div>Thanks in advance,</div><div>Pierre</div><div><div class="gmail-m_-4780025534987259656h5"><div><br><blockquote type="cite"><div>On 3 May 2018, at 5:07 PM, Hong <<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>> wrote:</div><br class="gmail-m_-4780025534987259656m_-2779756454327910987Apple-interchange-newline"><div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Pierre :</div><div class="gmail_quote">The code crashes inside</div><div class="gmail_quote">if (scall == MAT_REUSE_MATRIX) {</div><div class="gmail_quote">...</div><div class="gmail_quote">} </div><div class="gmail_quote">which assumes both original matrix and its submatrices data structures remain unchanged.</div><div class="gmail_quote">We did not anticipate the usage of looping<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
3) change the sparsity pattern of the Mat for one of the field<br>
4) do a KSPSolve<br></blockquote><div> </div><div>Can you send us a stand-alone code for this type of application.</div><div>We will investigate how to support it.</div><div><br></div><div>Hong</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I cannot change this calling sequence, because I’d like to nest 3) and 4) inside a loop.<br>
However, I end up with this error during 4):<br>
[0]PETSC ERROR: Nonconforming object sizes<br>
[0]PETSC ERROR: Cannot reuse matrix. wrong no of nonzeros<br>
[0]PETSC ERROR: Petsc Development GIT revision: v3.9-13-g05d412bc58  GIT Date: 2018-04-09 08:39:52 -0500<br>
<br>
[0]PETSC ERROR: #1 MatCreateSubMatrix_SeqAIJ() line 2330 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/mat/impls/ai<wbr>j/seq/aij.c<br>
[0]PETSC ERROR: #2 MatCreateSubMatrix_MPIAIJ_Same<wbr>RowColDist() line 3184 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/mat/impls/ai<wbr>j/mpi/mpiaij.c<br>
[0]PETSC ERROR: #3 MatCreateSubMatrix_MPIAIJ() line 3310 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/mat/impls/ai<wbr>j/mpi/mpiaij.c<br>
[0]PETSC ERROR: #4 MatCreateSubMatrix() line 7813 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/mat/interfac<wbr>e/matrix.c<br>
[0]PETSC ERROR: #5 PCSetUp_FieldSplit() line 564 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/ksp/pc/impls<wbr>/fieldsplit/fieldsplit.c<br>
[0]PETSC ERROR: #6 PCSetUp() line 923 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/ksp/pc/inter<wbr>face/precon.c<br>
[0]PETSC ERROR: #7 PCApply() line 453 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/ksp/pc/inter<wbr>face/precon.c<br>
[0]PETSC ERROR: #8 KSP_PCApply() line 276 in /Users/jolivet/repo/petsc/incl<wbr>ude/petsc/private/kspimpl.h<br>
[0]PETSC ERROR: #9 KSPFGMRESCycle() line 166 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/ksp/ksp/impl<wbr>s/gmres/fgmres/fgmres.c<br>
[0]PETSC ERROR: #10 KSPSolve_FGMRES() line 291 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/ksp/ksp/impl<wbr>s/gmres/fgmres/fgmres.c<br>
[0]PETSC ERROR: #11 KSPSolve() line 669 in /Users/jolivet/Documents/repos<wbr>itories/petsc/src/ksp/ksp/inte<wbr>rface/itfunc.c<br>
<br>
Is this the expected behaviour? Would you need a MWE?<br>
What I find funny is that if I turn -info on, then just before the error, I get:<br>
[0] PCSetUp(): Setting up PC with different nonzero pattern<br>
So why is it trying to reuse the matrix? I am missing a call to some function?<br>
<br>
Thanks in advance,<br>
Pierre</blockquote></div><br></div></div>
</div></blockquote></div><br></div></div></div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div></div></div>