<div dir="ltr">I agree Barry - we got a bit sidetracked here.<div><br></div><div>We were debating the relative merits of doing that (computing a Jacobian with every residual) and musing a bit about the possibility of whether or not SNES could be enhanced to make this more efficient.</div><div><br></div><div>I think I've been convinced that it's not worth pursuing the SNES enhancement... the upside is limited and the complexity is not worth it.</div><div><br></div><div>Derek</div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Dec 12, 2016 at 12:46 PM Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br class="gmail_msg">
> On Dec 11, 2016, at 10:58 PM, Derek Gaston <<a href="mailto:friedmud@gmail.com" class="gmail_msg" target="_blank">friedmud@gmail.com</a>> wrote:<br class="gmail_msg">
><br class="gmail_msg">
> A quick note: I'm not hugely invested in this idea... I'm just talking it out since I started it. The issues might outweigh potential gains...<br class="gmail_msg">
><br class="gmail_msg">
> On Sun, Dec 11, 2016 at 4:02 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" class="gmail_msg" target="_blank">knepley@gmail.com</a>> wrote:<br class="gmail_msg">
> I consider this bad code management more than an analytical case for the technique, but I can see the point.<br class="gmail_msg">
><br class="gmail_msg">
> Can you expand on that? Do you believe automatic differentiation in general to be "bad code management"?<br class="gmail_msg">
><br class="gmail_msg">
> - Extremely complex, heavy shape function evaluation (think super high order with first and second derivatives needing to be computed)<br class="gmail_msg">
><br class="gmail_msg">
> I honestly do not understand this one. Maybe I do not understand high order since I never use it. If I want to compute an integral, I have<br class="gmail_msg">
> the basis functions tabulated. I understand that for high order, you use a tensor product evaluation, but you still tabulate in 1D. What is<br class="gmail_msg">
> being recomputed here?<br class="gmail_msg">
><br class="gmail_msg">
> In unstructured mesh you still have to compute the reference->physical map for each element and map all of the gradients/second derivatives to physical space. This can be quite expensive if you have a lot of shape functions and a lot of quadrature points.<br class="gmail_msg">
><br class="gmail_msg">
> Sometimes we even have to do this step twice: once for the un-deformed mesh and once for the deformed mesh... on every element.<br class="gmail_msg">
><br class="gmail_msg">
><br class="gmail_msg">
> - Extremely heavy material property computations<br class="gmail_msg">
><br class="gmail_msg">
> Yes. I have to think about this one more.<br class="gmail_msg">
><br class="gmail_msg">
> - MANY coupled variables (we've run thousands).<br class="gmail_msg">
><br class="gmail_msg">
> Ah, so you are saying that the process of field evaluation at the quadrature points is expensive because you have so many fields.<br class="gmail_msg">
> It feels very similar to the material case, but I cannot articulate why.<br class="gmail_msg">
><br class="gmail_msg">
> It is similar: it's all about how much information you have to recompute at each quadrature point. I was simply giving different scenarios for why you could end up with heavy calculations at each quadrature point that feed into both the Residual and Jacobian calculations.<br class="gmail_msg">
><br class="gmail_msg">
> I guess my gut says that really expensive material properties,<br class="gmail_msg">
> much more expensive than my top level model, should be modeled by something simpler at that level. Same feeling for using<br class="gmail_msg">
> thousands of fields.<br class="gmail_msg">
><br class="gmail_msg">
> Even if you can find something simpler it's good to be able to solve the expensive one to verify your simpler model. Sometimes the microstructure behavior is complicated enough that it's quite difficult to wrap up in a simpler model or (like you said) it won't be clear if a simpler model is possible without doing the more expensive model first.<br class="gmail_msg">
><br class="gmail_msg">
> We really do have models that require thousands (sometimes tens of thousands) of coupled PDEs. Reusing the field evaluations for both the residual and Jacobian could be a large win.<br class="gmail_msg">
><br class="gmail_msg">
> 1) I compute a Jacobian with every residual. This sucks because line search and lots of other things use residuals.<br class="gmail_msg">
><br class="gmail_msg">
> 2) I compute a residual with every Jacobian. This sound like it could work because I compute both for the Newton system, but here I<br class="gmail_msg">
> am reusing the residual I computed to check the convergence criterion.<br class="gmail_msg">
><br class="gmail_msg">
> Can you see a nice way to express Newton for this?<br class="gmail_msg">
><br class="gmail_msg">
> You can see my (admittedly stupidly simple) Newton code that works this way here: <a href="https://github.com/friedmud/MOOSE.jl/blob/master/src/solvers/JuliaDenseNonlinearImplicitSolver.jl#L42" rel="noreferrer" class="gmail_msg" target="_blank">https://github.com/friedmud/MOOSE.jl/blob/master/src/solvers/JuliaDenseNonlinearImplicitSolver.jl#L42</a><br class="gmail_msg">
><br class="gmail_msg">
> Check the assembly code here to see how both are computed simultaneously: <a href="https://github.com/friedmud/MOOSE.jl/blob/master/src/solvers/Assembly.jl#L59" rel="noreferrer" class="gmail_msg" target="_blank">https://github.com/friedmud/MOOSE.jl/blob/master/src/solvers/Assembly.jl#L59</a><br class="gmail_msg">
><br class="gmail_msg">
> Lack of line search makes it pretty simple. However, even with this simple code I end up wasting one extra Jacobian evaluation once the convergence criteria has been reached. Whether or not that is advantageous depends on the relative tradeoffs of reusable element computations vs Jacobian calculation and how many nonlinear iterations you do (if you're only doing one nonlinear iteration every timestep then you're wasting 50% of your total Jacobian calculation time).<br class="gmail_msg">
><br class="gmail_msg">
> For a full featured solver you would definitely also want to have the ability to compute a residual, by itself, when you want... for things like line search.<br class="gmail_msg">
><br class="gmail_msg">
> You guys have definitely thought a lot more about this than I have... I'm just spitballing here... but it does seem like having an optional interface for computing a combined residual/Jacobian could save some codes a significant amount of time.<br class="gmail_msg">
><br class="gmail_msg">
> This isn't a strong feeling of mine though. I think that for now the way I'll do it is simply to "waste" a residual calculation when I need a Jacobian :-)<br class="gmail_msg">
<br class="gmail_msg">
Derek,<br class="gmail_msg">
<br class="gmail_msg">
I don't understand this long conversation. You can compute the Jacobian with the Function evaluation. There is no need for a special interface, just compute it! Then provide a dummy FormJacobian that does nothing. Please let us know if this does not work. (Yes you will need to keep the Jacobian matrix inside the FormFunction context so you have access to it but that is no big deal.)<br class="gmail_msg">
<br class="gmail_msg">
Barry<br class="gmail_msg">
<br class="gmail_msg">
><br class="gmail_msg">
> Derek<br class="gmail_msg">
<br class="gmail_msg">
</blockquote></div>