[petsc-users] Simultaneously compute Residual+Jacobian in SNES

[Jed Brown]

I was responding to your statement about an AD system being unable to compute a residual without also computing a Jacobian. Beyond that, it's just about how much intermediate work could be reused.

I still don't understand your many field case. The geometric overhead is amortized and the Jacobian contribution typically grows quadratically in the number of fields, so I would expect it to increase the Jacobian:residual cost ratio, not reduce it.

On December 12, 2016 8:36:09 AM PST, Derek Gaston <friedmud at gmail.com> wrote:
>On Mon, Dec 12, 2016 at 12:36 AM Jed Brown <jed at jedbrown.org> wrote:
>
>> > Can you expand on that?  Do you believe automatic differentiation
>in
>> > general to be "bad code management"?
>>
>> AD that prevents calling the non-AD function is bad AD.
>>
>
>That's not exactly the problem.  Even if you can call an AD and a
>non-AD
>residual... you still have to compute two residuals to compute a
>residual
>and a Jacobian separately when using AD.
>
>It's not the end of the world... but it was something that prompted me
>to
>ask the question.
>
>
>> Are all the fields in unique function spaces that need different
>> transforms or different quadratures?  If not, it seems like the
>presence
>> of many fields would already amortize the geometric overhead of
>visiting
>> an element.
>>
>
>These were two separate examples.  Expensive shape functions, by
>themselves, could warrant computing the residual and Jacobian
>simultaneously.  Also: many variables, by themselves, could do the
>same.
>
>
>> Alternatively, you could cache the effective material coefficient
>(and its
>> gradient) at each quadrature point during residual evaluation, thus
>> avoiding a re-solve when building the Jacobian.
>
>
>I agree with this.  We have some support for it in MOOSE now... and
>more
>plans for better support in the future.  It's a classic time/space
>tradeoff.
>
>
>> I would recommend that unless you know that line searches are rare.
>>
>
>BTW: Many (most?) of our most complex applications all _disable_ line
>search.  Over the years we've found line search to be more of a
>hindrance
>than a help.  We typically prefer using some sort of "physics based"
>damped
>Newton.
>
>
>> It is far more common that the Jacobian is _much_ more expensive than
>> the residual, in which case the mere possibility of a line search (or
>of
>> converging) would justify deferring the Jacobian.  I think it's much
>> better to make residuals and Jacobians fast independently, then
>perhaps
>> make the residual do some cheap caching, and worry about
>> second-guessing Newton only as a last resort.
>
>
>I think I agree.  These are definitely "fringe" cases... for most
>applications Jacobians are _way_ more expensive.
>
>
>> That said, I have no doubt that we could
>> demonstrate some benefit to using heuristics and a relative cost
>model
>> to sometimes compute residuals and Jacobians together.  It just isn't
>> that interesting and I think the gains are likely small and will
>> generate lots of bikeshedding about the heuristic.
>>
>
>I agree here too.  It could be done... but I think you've convinced me
>that
>it's not worth the trouble :-)
>
>Thanks for the discussion everyone!
>
>Derek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20161212/cc218a70/attachment-0001.html>