<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, Dec 12, 2017 at 10:39 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-">On Tue, Dec 12, 2017 at 12:26 PM, Alexander Lindsay <span dir="ltr"><<a href="mailto:alexlindsay239@gmail.com" target="_blank">alexlindsay239@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Ok, I'm going to go back on my original statement...the physics being run here is a sub-set of a much larger set of physics; for the current set the hand-coded Jacobian actually appears to be quite good.<div><br></div><div>With hand-coded Jacobian, -pc_type lu, the convergence is perfect:</div><div><br></div><div><div> 0 Nonlinear |R| = 2.259203e-02</div><div> 0 Linear |R| = 2.259203e-02</div><div> 1 Linear |R| = 1.129089e-10</div><div> 1 Nonlinear |R| = 6.295583e-11</div></div><div><br></div><div>So yea I guess at this point I'm just curious about the different behavior between `-snes_fd` and `-snes_fd -snes_mf_operator`. Does the hand-coded result change your opinion Matt that the rules for FormFunction/Jacobian might be being violated?</div><div><br></div><div>I understand that a finite difference approximation of the true Jacobian <i>is an approximation. </i>However, in the absence of possible complications like Matt suggested where an on-the-fly calculation might stand a better chance of capturing the behavior, I would expect both `-snes_mf_operator -snes_fd` and `-snes_fd` to suffer from the same approximations, right?</div></div></blockquote><div><br></div></span><div>There are too many things that do not make sense:</div><div><br></div><div>1) How could LU be working with -snes_mf_operator?</div><div><br></div><div> What operator are you using here. The hand-coded Jacobian?</div></div></div></div></blockquote><div><br></div><div>Why wouldn't LU work with -snes_mf_operator? My understanding is that LU is configured to operate interchangeably with iterative preconditioners. </div><div><br></div><div>Jacobian:</div><div><br></div><div><div>(lldb) call MatView(snes->jacobian, 0)</div><div>Mat Object: 1 MPI processes</div><div> type: mffd</div><div> Matrix-free approximation:</div><div> err=1.49012e-08 (relative error in function evaluation)</div><div> The compute h routine has not yet been set</div></div><div><br></div><div>Preconditioner is computed with with SNESComputeJacobianDefault</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>2) How can LU take more than one iterate?</div><span class="gmail-"><div><br></div><div><span style="font-size:12.8px"><span class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-im"><div>0 Nonlinear |R| = 2.259203e-02</div><div> 0 Linear |R| = 2.259203e-02</div></span></span><div style="font-size:12.8px"> 1 Linear |R| = 2.258733e-02</div><div style="font-size:12.8px"> 2 Linear |R| = 3.103342e-06</div><div style="font-size:12.8px"> 3 Linear |R| = 6.779865e-12</div></div><div><br></div></span><div>That seems to say that we are not solving a linear system for some reason.</div></div></div></div></blockquote><div><br></div><div>I'm curious about the same thing :-) Perhaps because B^-1 is not a perfect inverse of the action of J on x in the matrix-free case? If I actually run with matrix-free J and <b>hand-coded</b> B, I do get one linear iteration:</div><div><br></div><div><div>SMP jacobian PJFNK, -pc_type lu:</div><div> 0 Nonlinear |R| = 2.259203e-02</div><div> 0 Linear |R| = 2.259203e-02</div><div> 1 Linear |R| = 2.857149e-11</div><div> 1 Nonlinear |R| = 1.404604e-11</div><div> Solve Converged!</div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div> 3) Why would -snes_fd be different from a hand-coded Jacobian?</div><div><br></div><div> Did you try -snes_type test?</div></div></div></div></blockquote><div><br></div><div>The matrix ratio norm of the difference is on the order of 1e-6, which isn't perfect but here the hand-coded performs better than the finite-differenced Jacobian.</div><div><br></div><div>Alex </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div> Matt</div><div><div class="gmail-h5"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_extra"><div class="gmail_quote">On Tue, Dec 12, 2017 at 9:43 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Tue, Dec 12, 2017 at 11:30 AM, Alexander Lindsay <span dir="ltr"><<a href="mailto:alexlindsay239@gmail.com" target="_blank">alexlindsay239@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr" style="font-size:12.8px">I'm not using any hand-coded Jacobians.</div></div></blockquote><div><br></div></span><div>This looks to me like the rules for FormFunction/Jacobian() are being broken. If the residual function</div><div>depends on some third variable, and it changes between calls independent of the solution U, then</div><div>the stored Jacobian could look wrong, but one done every time on the fly might converge.</div><div><br></div><div> Matt</div><div><div class="gmail-m_-7344126864435285382gmail-m_2954631125155224043h5"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr" style="font-size:12.8px"><div>Case 1 options: -snes_fd -pc_type lu</div><div><br></div><div><span><span class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-im"><div>0 Nonlinear |R| = 2.259203e-02</div><div> 0 Linear |R| = 2.259203e-02</div></span></span><div> 1 Linear |R| = 7.821248e-11</div><span><span class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-im"><div> 1 Nonlinear |R| = 2.258733e-02</div><div> 0 Linear |R| = 2.258733e-02</div></span></span><div> 1 Linear |R| = 5.277296e-11</div><span><span class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-im"><div> 2 Nonlinear |R| = 2.258733e-02</div><div> 0 Linear |R| = 2.258733e-02</div></span></span><div> 1 Linear |R| = 5.993971e-11</div><span><span class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-im">Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH iterations 2</span></span></div><div><br></div><div>Case 2 options: -snes_fd -snes_mf_operator -pc_type lu</div><div><br></div><div><span><span class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-im"><div> 0 Nonlinear |R| = 2.259203e-02</div><div> 0 Linear |R| = 2.259203e-02</div></span></span><div> 1 Linear |R| = 2.258733e-02</div><div> 2 Linear |R| = 3.103342e-06</div><div> 3 Linear |R| = 6.779865e-12</div><div> 1 Nonlinear |R| = 7.497740e-06</div><div> 0 Linear |R| = 7.497740e-06</div><div> 1 Linear |R| = 8.265413e-12</div><div> 2 Nonlinear |R| = 7.993729e-12</div><div>Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 2</div></div></div><div class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-yj6qo gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-ajU" style="margin:2px 0px 0px;font-size:12.8px"><div id="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-:2o0" class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-ajR"><img class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026m_-7017120415024906329gmail-ajT" src="https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif" style="opacity: 0.3;"></div></div></div><div class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026HOEnZb"><div class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 12, 2017 at 9:12 AM, zakaryah . <span dir="ltr"><<a href="mailto:zakaryah@gmail.com" target="_blank">zakaryah@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-size:small">When you say "Jacobians are bad" and "debugging the Jacobians", do you mean that the hand-coded Jacobian is wrong? In that case, why would you be surprised that the finite difference Jacobians, which are "correct" to approximation error, perform better? Otherwise, what does "Jacobians are bad" mean - ill-conditioned? Singular? Not symmetric? Not positive definite?</div></div>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><br><br clear="all"><span class="gmail-m_-7344126864435285382gmail-HOEnZb"><font color="#888888"><span><div><br></div>-- <br><div class="gmail-m_-7344126864435285382gmail-m_2954631125155224043m_4114183395812912026gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.caam.rice.edu/~mk51/" target="_blank">https://www.cse.buffalo.edu/~k<wbr>nepley/</a><br></div></div></div></div></div>
</span></font></span></div></div>
</blockquote></div><br></div>
</blockquote></div></div></div><div><div class="gmail-h5"><br><br clear="all"><div><br></div>-- <br><div class="gmail-m_-7344126864435285382gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.caam.rice.edu/~mk51/" target="_blank">https://www.cse.buffalo.edu/~<wbr>knepley/</a><br></div></div></div></div></div>
</div></div></div></div>
</blockquote></div><br></div></div>