[petsc-users] Simultaneously compute Residual+Jacobian in SNES

[Patrick Sanan]

On Mon, Dec 12, 2016 at 6:33 PM, Jed Brown <jed at jedbrown.org> wrote:
> Patrick Sanan <patrick.sanan at gmail.com> writes:
>> Maybe a good way to proceed is to have the "both at once computation"
>> explicitly available in PETSc's Newton or other solvers that "know"
>> that they can gain efficiency by computing both at once. That is, the
>> user is always required to register a callback to form the residual,
>> and may optionally register a function to compute (only) the Jacobian
>> and/or another function to compute the combined Jacobian/Residual.
>> Newton can look for the combined function at the times when it would
>> help, and if nothing was registered would fall back to the current
>> technique of calling separate residual and Jacobian routines. This
>> would preserve the benefits of the current approach, allow
>> optimizations of the kind mentioned, and hopefully remain
>> somewhat-maintainable; in particular, this doesn't require the user to
>> be promised anything about the order in which Jacobians and residuals
>> are computed.
>
> How would the solver decide?  It requires a model for the relative costs
> of {residual, Jacobian, residual+Jacobian}.  Where would the SNES learn
> that?
>
> I'm not saying the above is wrong, but it adds complexity and wouldn't
> be useful without the cost model.  An alternative would be for the
> user's residual to be able to query the SNES's estimated probability
> that a Jacobian will be needed.  So if the SNES has observed quadratic
> convergence, it might be quite confident (unless it expects this
> residual to meet the convergence criteria).  The advantage of this
> scheme is that it applies to partial representations of the Jacobian
> (like storing the linearization at quadrature points, versus assembling
> the sparse matrix).

I did have an error in my thinking, which is that I forgot about the
fact that you typically have the residual already at the point where
the Jacobian is computed, so would would have to be using a method
(like the cartoon in my mind when I wrote the above) that didn't
involve a linesearch (as Derek also described) or as you suggest,
somehow had a guess about when the linesearch would terminate.