<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 29, 2015 at 11:42 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Tue, Sep 29, 2015 at 10:28 AM, Gil Forsyth <span dir="ltr"><<a href="mailto:gforsyth@gwu.edu" target="_blank">gforsyth@gwu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Hi all, </div><div><br></div>I've been having some trouble with what should be a relatively simple update to an immersed boundary CFD solver from PETSc 3.5.4 to 3.6.1<div><br></div><div>I'm getting indefinite PC errors for a simple lid-driven cavity test problem, 32x32 at Re 100</div><div><br></div><div>Under PETSc 3.5.4 using KSPSetNullSpace we used the following to set the null space. This is for a 2D Poisson system with no immersed boundary and so the null space is the constant vector.<br><div><br></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div><div> MatNullSpace nsp;</div></div></div><div><div><span style="white-space:pre-wrap"> </span>ierr = MatNullSpaceCreate(PETSC_COMM_WORLD, PETSC_TRUE, 0, NULL, &nsp); CHKERRQ(ierr);</div></div><div><div><span style="white-space:pre-wrap"> </span>ierr = KSPSetNullSpace(ksp2, nsp); CHKERRQ(ierr);</div></div><div><div> ierr = MatNullSpaceDestroy(&nsp); CHKERRQ(ierr);</div></div></blockquote></div></blockquote><div><br></div></span><div>Clearly this has to happen in the reverse order, since ksp2 would not be created yet.</div><div><br></div><div>For questions about solvers, we HAVE to see the complete output of -ksp_view so we</div><div>know what we are dealing with. Its also nice to have -ksp_monitor_true_residual -ksp_converged_reason</div><div><br></div><div> Matt</div></div></div></div></blockquote><div><br></div><div>Yes -- sorry, those are both in inline files and are called in the reverse order that I wrote them out. </div><div><br></div><div>I've attached the output of </div><div><br></div><div><div>$PETIBM_DIR/petibm3.6/bin/petibm2d -directory . -poisson_pc_type gamg -poisson_pc_gamg_type agg -poisson_gamg_agg_nsmooths 1 -poisson_ksp_view -poisson_ksp_monitor_true_residual -poisson_ksp_converged_reason > kspview.log</div></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>And then setup the KSP with</div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div> ierr = KSPCreate(PETSC_COMM_WORLD, &ksp2); CHKERRQ(ierr);</div></div><div><div> ierr = KSPSetOptionsPrefix(ksp2, "poisson_"); CHKERRQ(ierr);</div></div><div><div> ierr = KSPSetOperators(ksp2, QTBNQ, QTBNQ); CHKERRQ(ierr);</div></div><div><div> ierr = KSPSetInitialGuessNonzero(ksp2, PETSC_TRUE); CHKERRQ(ierr);</div></div><div><div> ierr = KSPSetType(ksp2, KSPCG); CHKERRQ(ierr);</div></div><div><div> ierr = KSPSetReusePreconditioner(ksp2, PETSC_TRUE); CHKERRQ(ierr);</div></div><div><div> ierr = KSPSetFromOptions(ksp2); CHKERRQ(ierr);</div></div></blockquote><div><br></div>The matrix QTBNQ does not change, only the rhs of the system is updated.<br><div><br></div><div>We run this with `-pc_type gamg -pc_gamg_type agg -pc_gamg_agg_nsmooths 1` and everything seems to work as expected. </div><div><br></div><div>Under PETSc 3.6.1, we change only the KSPSetNullSpace line, to</div><div><br></div><div> ierr = MatSetNullSpace(QTBNQ, nsp); CHKERRQ(ierr);</div><div><br></div><div>and the same code diverges after 1 timestep and returns a -8 KSP_DIVERGED_INDEFINITE_PC</div><div><br></div><div>This is weird, especially because if we change nsmooths to 2, it runs for 264 timesteps and the returns the same error. But we have explicitly set KSPSetReusePreconditioner so it should be using the same PC, right?</div><div><br></div><div>Change nsmooths to 3 and it again diverges after 1 timestep. </div><div><br></div><div>Change nsmooths to 4 and it runs to completion. </div><div><br></div><div>It seems like either gamg's behavior has changed, or that KSPSetNullSpace was doing something implicitly that we now need to do explicitly in addition to MatSetNullSpace? </div><div><br></div><div>Thanks, </div><span><font color="#888888"><div>Gil Forsyth</div></font></span></div>
</blockquote></span></div><span class=""><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</font></span></div></div>
</blockquote></div><br></div></div>