<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hong,<div class="">Thank you for the patch.</div><div class="">Our application is not crashing anymore.</div><div class="">Hopefully this will be merged quickly in master.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Pierre<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 7 May 2018, at 10:07 PM, Hong <<a href="mailto:hzhang@mcs.anl.gov" class="">hzhang@mcs.anl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline" class="">Pierre:</span><div class="">I added this support </div><div class=""><a href="https://bitbucket.org/petsc/petsc/commits/ef7efd37182b12f7b7ab7c34390aae8e592a83c2" class="">https://bitbucket.org/petsc/petsc/commits/ef7efd37182b12f7b7ab7c34390aae8e592a83c2</a></div><div class=""><br class=""></div><div class="">It is in branch hzhang/fix-pc-fieldsplit-reuse.</div><div class="">You may give it a try. Once it passed all our regression tests, it will be merged to petsc-master branch.</div><div class=""><br class=""></div><div class="">Your modified ex42.c is very helpful, thanks!</div><div class="">Hong<br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Fri, May 4, 2018 at 10:15 AM, Hong <span dir="ltr" class=""><<a href="mailto:hzhang@mcs.anl.gov" target="_blank" class="">hzhang@mcs.anl.gov</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote">Pierre:</div><div class="gmail_quote">Thanks for creating this use case. I'll check it and get back to you sometime next week.</div><div class="gmail_quote"><span class="gmail-HOEnZb"><font color="#888888" class="">Hong</font></span><div class=""><div class="gmail-h5"><br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class="">Hong,<div class="">Thanks for your reply.</div><div class="">It’s not so minimalist, but at least it is based on one of your example.</div><div class=""><div class="">diff --git a/src/ksp/ksp/examples/tutoria<wbr class="">ls/ex42.c b/src/ksp/ksp/examples/tutoria<wbr class="">ls/ex42.c</div><div class="">index 8be4389a2d..520c82569a 100644</div><div class="">--- a/src/ksp/ksp/examples/tutoria<wbr class="">ls/ex42.c</div><div class="">+++ b/src/ksp/ksp/examples/tutoria<wbr class="">ls/ex42.c</div><div class="">@@ -1953,6 +1953,15 @@ static PetscErrorCode solve_stokes_3d_coupled(PetscI<wbr class="">nt mx,PetscInt my,PetscInt m</div><div class=""> }</div><div class=""> }</div><div class=""> ierr = KSPSolve(ksp_S,f,X);CHKERRQ(ie<wbr class="">rr);</div><div class="">+ PetscViewerPushFormat(PETSC_V<wbr class="">IEWER_STDOUT_WORLD, PETSC_VIEWER_ASCII_INFO);</div><div class="">+ MatView(A, PETSC_VIEWER_STDOUT_WORLD);</div><div class="">+ Mat C;</div><div class="">+ MatMatMult(A, A, MAT_INITIAL_MATRIX, PETSC_DEFAULT, &C);</div><div class="">+ MatView(C, PETSC_VIEWER_STDOUT_WORLD);</div><div class="">+ PetscViewerPopFormat(PETSC_VI<wbr class="">EWER_STDOUT_WORLD);</div><div class="">+ ierr = KSPSetOperators(ksp_S,C,C);CHK<wbr class="">ERRQ(ierr);</div><div class="">+ ierr = KSPSolve(ksp_S,f,X);CHKERRQ(ie<wbr class="">rr);</div><div class="">+ MatDestroy(&C);</div><div class=""><br class=""></div><div class=""> ierr = PetscOptionsGetBool(NULL,NULL,<wbr class="">"-write_pvts",&write_output,NU<wbr class="">LL);CHKERRQ(ierr);</div><div class=""> if (write_output) {ierr = DAView3DPVTS(da_Stokes,X,"up")<wbr class="">;CHKERRQ(ierr);}</div><div class=""><br class=""></div><div class="">$ mpirun -np 1 ./ex42 -mx 2 -my 2 -mz 2 -stokes_pc_type fieldsplit</div><div class="">This should give you this exact error.</div><div class="">Do you know of a way to bypass the error by flushing/destroying the PC while still keeping the options for the following KSPSolve?</div><div class=""><br class=""></div><div class="">Thanks in advance,</div><div class="">Pierre</div><div class=""><div class="gmail-m_-4780025534987259656h5"><div class=""><br class=""><blockquote type="cite" class=""><div class="">On 3 May 2018, at 5:07 PM, Hong <<a href="mailto:hzhang@mcs.anl.gov" target="_blank" class="">hzhang@mcs.anl.gov</a>> wrote:</div><br class="gmail-m_-4780025534987259656m_-2779756454327910987Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote">Pierre :</div><div class="gmail_quote">The code crashes inside</div><div class="gmail_quote">if (scall == MAT_REUSE_MATRIX) {</div><div class="gmail_quote">...</div><div class="gmail_quote">} </div><div class="gmail_quote">which assumes both original matrix and its submatrices data structures remain unchanged.</div><div class="gmail_quote">We did not anticipate the usage of looping<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br class="">
3) change the sparsity pattern of the Mat for one of the field<br class="">
4) do a KSPSolve<br class=""></blockquote><div class=""> </div><div class="">Can you send us a stand-alone code for this type of application.</div><div class="">We will investigate how to support it.</div><div class=""><br class=""></div><div class="">Hong</div><div class=""><br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I cannot change this calling sequence, because I’d like to nest 3) and 4) inside a loop.<br class="">
However, I end up with this error during 4):<br class="">
[0]PETSC ERROR: Nonconforming object sizes<br class="">
[0]PETSC ERROR: Cannot reuse matrix. wrong no of nonzeros<br class="">
[0]PETSC ERROR: Petsc Development GIT revision: v3.9-13-g05d412bc58 GIT Date: 2018-04-09 08:39:52 -0500<br class="">
<br class="">
[0]PETSC ERROR: #1 MatCreateSubMatrix_SeqAIJ() line 2330 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/mat/impls/ai<wbr class="">j/seq/aij.c<br class="">
[0]PETSC ERROR: #2 MatCreateSubMatrix_MPIAIJ_Same<wbr class="">RowColDist() line 3184 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/mat/impls/ai<wbr class="">j/mpi/mpiaij.c<br class="">
[0]PETSC ERROR: #3 MatCreateSubMatrix_MPIAIJ() line 3310 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/mat/impls/ai<wbr class="">j/mpi/mpiaij.c<br class="">
[0]PETSC ERROR: #4 MatCreateSubMatrix() line 7813 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/mat/interfac<wbr class="">e/matrix.c<br class="">
[0]PETSC ERROR: #5 PCSetUp_FieldSplit() line 564 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/ksp/pc/impls<wbr class="">/fieldsplit/fieldsplit.c<br class="">
[0]PETSC ERROR: #6 PCSetUp() line 923 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/ksp/pc/inter<wbr class="">face/precon.c<br class="">
[0]PETSC ERROR: #7 PCApply() line 453 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/ksp/pc/inter<wbr class="">face/precon.c<br class="">
[0]PETSC ERROR: #8 KSP_PCApply() line 276 in /Users/jolivet/repo/petsc/incl<wbr class="">ude/petsc/private/kspimpl.h<br class="">
[0]PETSC ERROR: #9 KSPFGMRESCycle() line 166 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/ksp/ksp/impl<wbr class="">s/gmres/fgmres/fgmres.c<br class="">
[0]PETSC ERROR: #10 KSPSolve_FGMRES() line 291 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/ksp/ksp/impl<wbr class="">s/gmres/fgmres/fgmres.c<br class="">
[0]PETSC ERROR: #11 KSPSolve() line 669 in /Users/jolivet/Documents/repos<wbr class="">itories/petsc/src/ksp/ksp/inte<wbr class="">rface/itfunc.c<br class="">
<br class="">
Is this the expected behaviour? Would you need a MWE?<br class="">
What I find funny is that if I turn -info on, then just before the error, I get:<br class="">
[0] PCSetUp(): Setting up PC with different nonzero pattern<br class="">
So why is it trying to reuse the matrix? I am missing a call to some function?<br class="">
<br class="">
Thanks in advance,<br class="">
Pierre</blockquote></div><br class=""></div></div>
</div></blockquote></div><br class=""></div></div></div></div></blockquote></div></div></div><br class=""></div></div>
</blockquote></div><br class=""></div></div></div>
</div></blockquote></div><br class=""></div></body></html>