[petsc-dev] Discussion about time-dependent optimization moved from PR

[Jed Brown]

Stefano Zampini <stefano.zampini at gmail.com> writes:

>> >
>> >
>> >> In case of multiple objectives, there may be a performance reason to
>> >> amortize evaluation of several at once, though the list interface is
>> >> convenient.  Consider common objectives being quantities like lift and
>> >> drag on different surfaces of a fluids simulation or stress/strain at
>> >> certain critical joints in a structure.  Although these have some
>> >> locality, it's reasonable to assume that state dependence will have
>> >> quickly become global, thus make no attempt to handle sparse
>> >> representations of the adjoint vectors lambda.
>> >>
>> >>
>> > I don't get this comment. Is it related with multi-objective optimization
>> > (e.g. Pareto)?
>>
>> Adjoints are usually preferred any time you are differentiating a small
>> number of output variables with respect to a large number of inputs.  It
>> could be for multi-objective optimization, but it's every bit as
>> relevant for physical sensitivities.
>>
>>
> If we are not talking about Pareto optimization, and thus we don't need a
> separate output from each function, then users can pass a single function
> that computes all the quantities they need at the same time. Anyway, I
> don't mind having a single callback for multiple functions.

I don't follow why multi-objective optimization would be any different.
Are you supposing that the different objective functions would be
implemented by different software packages?

>> >> How are parameters accessed in TSComputeRHSFunction?  It looks like
>> >> they're coming out of the context.  Why should this be different?  (If
>> >> parameters need to go into a Vec, we could do that, but it comes at a
>> >> readability and possibly parallel cost if the global Vec needs to be
>> >> communicated to local vectors.)
>> >>
>> >>
>> > design paramaters are fixed troughout an adjoint/tlm run. They can be
>> > communicated locally once at the beginning of the run.
>> > This is what TSSetUpFromDesign and TSSetSetUpFromDesign are supposed to
>> > handle, if I get your comment.
>>
>> My point is that users currently get design parameters out of the
>> context when evaluating their RHSFunction and friends.  If that is the
>> endorsed way to access design variables, then your new function doesn't
>> need to pass the vector.  If you need to pass the parameter vector in
>> that one function, instead of obtaining them from the context, then
>> you'd need to pass the parameter vector everywhere and discourage using
>> the context for active design variables.  I think there are merits to
>> both approaches, but it absolutely needs to be consistent.
>>
>
> So, to be consistent, we have to force users to perform an operation in a
> single way?
> Yes, TSSetUpFromDesign is among the last things I have added, and allows to
> update the application context (among other things, as it is very general).
> I can remove the parameter vector from TSEvalGradientDAE and
> TSEvalGradientIC; however, having these vectors there makes it clear that
> we allow non-linear dependency on the parameters too. 

There are a bunch of other places that allow nonlinear dependence on
parameters yet don't pass that parameter Vec, including
TSSetRHSFunction.  If it is passed explicitly in one such function, it
needs to be passed explicitly in all.

> I can add a comment on the man pages that the vectors are guaranteed
> to be the same one passed in my TSSetUpFromDesign, or remove
> them. Your call.

It's a recipe for confusion.  Either the parameters are never passed
explicitly or they are always passed explicitly and should not be stored
redundantly in the context, thus perhaps enabling some sort of higher
level analysis that dynamically changes parameter values.  I would go
with the former for now.

> Users can do anything they want with the forward model context, but they
> are not free to change the application context of the adjoints TS. Maybe
> this should be improved by adding and extra slot to AdjointTSCtx to carry
> over the  user context (for the adjoint I mean)?

Why do you need to own the application context for the adjoint TS,
versus using the context parameters to TSSetRHSJacobian, etc.

>> > https://bitbucket.org/petsc/petsc/src/c2e9112e7fdfd89985f9ffc4d68b0d
>> 46cf7cad52/src/ts/interface/tspdeconstrainedutils.c?at=
>> stefano_zampini%2Ffeature-continuousadjoint&fileviewer=file-view-default#
>> tspdeconstrainedutils.c-579
>> >
>> > Here is how ex23.c uses it
>> >
>> > https://bitbucket.org/petsc/petsc/src/c2e9112e7fdfd89985f9ffc4d68b0d
>> 46cf7cad52/src/ts/examples/tutorials/ex23.c?at=stefano_zampini%2Ffeature-
>> continuousadjoint&fileviewer=file-view-default#ex23.c-677
>>
>> And yet you redo the scatter here instead of using what you stuffed into
>> the context.  If you needed to redo it for correctness, you'd also need
>> to in every other function that accesses design parameters.
>>
> https://bitbucket.org/petsc/petsc/src/c2e9112e7fdfd89985f9ffc4d68b0d
>> 46cf7cad52/src/ts/examples/tutorials/ex23.c?at=stefano_zampini%2Ffeature-
>> continuousadjoint&fileviewer=file-view-default#ex23.c-274
>>
>>
> This is a leftover from a previous version of the code (there's also a
> comment) that was not using TSSetSetUpFromDesign and it's definitely not
> needed.

Comments that some lines of code can be deleted is a good sign that they
should be deleted.

>> >> > Both methods need the Jacobian of the DAE wrt the parameters: H
>> >> > TSAdjointSetRHSJacobian(), S TSSetGradientDAE()
>> >> >
>> >> > Initial condition dependence on the parameters is implicitly computed
>> in
>> >> > Hong's code (limited to linear dependence on all the variables);
>> >>
>> >> How so?  Once the user gets \lambda(time=0), they can apply the chain
>> >> rule to produce any dependency on the parameter vector?
>> >>
>> >>  Yes, the chain rule is implemented here
>> >
>> > https://bitbucket.org/petsc/petsc/src/c2e9112e7fdfd89985f9ffc4d68b0d
>> 46cf7cad52/src/ts/interface/tspdeconstrainedutils.c?at=
>> stefano_zampini%2Ffeature-continuousadjoint&fileviewer=file-view-default#
>> tspdeconstrainedutils.c-254
>>
>> I know you have a callback for it, but Hong's interface is plenty
>> functional for such systems, they just call that derivative instead of
>> writing and registering a function to do that.
>>
>
> I'm not saying HOng's interface is not functional. 

I was responding to your statement that "Initial condition dependence on
the parameters is limited to linear dependence on all the variables" in
his approach.  General dependence is supported, the user is just
expected to call it themselves rather than registering a callback that
TS calls.

> The initial condition gradients allow to automatize the process in
> TSComputeObjectiveAndGradient(), TSComputeHessian(), and
> MatMult_Propagator(). Anyway, the lambda variables are not modified by
> AdjointTSComputeFinalGradient() (that adds the IC dependency) and
> users can do whatever they want with them.
>
> I know you don't like TSComputeObjectiveAndGradient(), 

I don't have a problem with the concept but don't want to rely on it.
I'd rather discuss how to unify interfaces for discrete and continuous
adjoints and get a PR for it merged before adding higher level
interfaces.

> but it's a code that anyway users have to write to compute a gradient
> of the DAE not arising from a PDAE. How can this be automatized for
> PDAEs? Should we store the AdjointTS inside the model TS and use
> TSGetAdjointTS(ts,&ts->adjts) with

I see reasons for the adjoint TS to hold a reference to the forward TS,
but not vice-versa.  What is the use case for your proposal below?

> TSGetAdjointTS(TS f,TS* a) {
> if (!f->adjtts) TSCreateAdjointsTS(f,&f->adjts);
> *a = f->adjts:
> }
>
> and the corresponding Setter to allow users to do
>
> TSCreateAdjointTS(ts,&atts)
> TSSetRHSJacobian(ats,...)
> TSSetAdjointTS(ts,ats)
>
> or
>
> TSGetAdjointTS(ts,&atts)
> TSSetRHSJacobian(ats,...)
>
>
> -- 
> Stefano