[petsc-users] Questions on TAO and gradient norm / inner products

[Matthew Knepley]

On Fri, Jul 12, 2024 at 7:25 AM Blauth, Sebastian <
sebastian.blauth at itwm.fraunhofer.de> wrote:

> Dear Barry,
>
>
>
> thanks for the clarification. Oh, that’s unfortunate, that not all TAO
> algorithms use the supplied matrix for the norm (and then probably also not
> for computing inner products in, e.g., the limited memory formulas).
>
>
>
> I fear that I don’t have sufficient time at the moment to make a MR. I
> could, however, provide some “minimal” example where the behavior is shown.
> However, that example would be using petsc4py as I am only familiar with
> that and I would use the fenics FEM package to define the matrices. Would
> this be okay?
>

Yes


> And if that’s the case, should I post the example here or at the petsc
> gitlab?
>

Either place is fine. Gitlab makes it easier to track.

  Thanks,

     Matt


> Best regards,
>
> Sebastian
>
>
>
> --
>
> Dr. Sebastian Blauth
>
> Fraunhofer-Institut für
>
> Techno- und Wirtschaftsmathematik ITWM
>
> Abteilung Transportvorgänge
>
> Fraunhofer-Platz 1, 67663 Kaiserslautern
>
> Telefon: +49 631 31600-4968
>
> sebastian.blauth at itwm.fraunhofer.de
>
> https://urldefense.us/v3/__https://www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxKWFVczr$ 
>
>
>
> *From:* Barry Smith <bsmith at petsc.dev>
> *Sent:* Tuesday, July 9, 2024 3:32 PM
> *To:* Blauth, Sebastian <sebastian.blauth at itwm.fraunhofer.de>; Munson,
> Todd <tmunson at mcs.anl.gov>; toby Isaac <toby.isaac at gmail.com>
> *Cc:* petsc-users at mcs.anl.gov
> *Subject:* Re: [petsc-users] Questions on TAO and gradient norm / inner
> products
>
>
>
>
>
>    From
>
>
>
> $ git grep TaoGradientNorm
>
> bound/impls/blmvm/blmvm.c:  PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
> bound/impls/blmvm/blmvm.c:    PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
> bound/impls/bnk/bnk.c:  PetscCall(*TaoGradientNorm*(tao, tao->gradient,
> NORM_2, &bnk->gnorm));
>
> bound/impls/bnk/bnk.c:          PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &bnk->gnorm));
>
> bound/impls/bnk/bnls.c:      PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &bnk->gnorm));
>
> bound/impls/bnk/bntl.c:      PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &bnk->gnorm));
>
> bound/impls/bnk/bntl.c:        PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &bnk->gnorm));
>
> bound/impls/bnk/bntr.c:        PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &bnk->gnorm));
>
> interface/taosolver.c:.seealso: [](ch_tao), `Tao`,
> `TaoGetGradientNorm()`, `*TaoGradientNorm*()`
>
> interface/taosolver.c:.seealso: [](ch_tao), `Tao`,
> `TaoSetGradientNorm()`, `*TaoGradientNorm*()`
>
> interface/taosolver.c:  *TaoGradientNorm* - Compute the norm using the
> `NormType`, the user has selected
>
> interface/taosolver.c:PetscErrorCode *TaoGradientNorm*(Tao tao, Vec
> gradient, NormType type, PetscReal *gnorm)
>
> unconstrained/impls/lmvm/lmvm.c:  PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
> unconstrained/impls/lmvm/lmvm.c:      PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
> unconstrained/impls/nls/nls.c:  PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
> unconstrained/impls/nls/nls.c:          PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
> unconstrained/impls/nls/nls.c:    PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
> unconstrained/impls/ntr/ntr.c:  PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
> unconstrained/impls/ntr/ntr.c:        PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
> unconstrained/impls/ntr/ntr.c:      PetscCall(*TaoGradientNorm*(tao,
> tao->gradient, NORM_2, &gnorm));
>
>
>
> it appears only some of the algorithm implementations use the norm you
> provide. While
>
>
>
>  git grep VecNorm
>
>
>
> indicates many places where it is not used. Likely some of the other
> algorithm implementations could be easily "fixed" to support by
>
> changing the norm computed. But I am not an expert on the algorithms and
> don't know if all algorithms can mathematically support a user provided
> norm.
>
>
>
> You are welcome to take a stab at making the change in an MR, or do you
> have a simple test problem with a mass matrix we can use to fix the
>
> "missing" implementations?
>
>
>
>   Barry
>
>
>
>
>
>
>
>
>
>
>
> On Jul 9, 2024, at 3:47 AM, Blauth, Sebastian <
> sebastian.blauth at itwm.fraunhofer.de> wrote:
>
>
>
> Hello,
>
>
>
> I have some questions regarding TAO and the use the gradient norm.
>
>
>
> First, I want to use a custom inner product for the optimization in TAO
> (for computing the gradient norm and, e.g., in the double loop of a
> quasi-Newton method). I have seen that there is the method
> TAOSetGradientNorm
> https://urldefense.us/v3/__https://petsc.org/release/manualpages/Tao/TaoSetGradientNorm/__;!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxDTBKkKq$  which seems
> to do this. According to the petsc4py docs
> https://urldefense.us/v3/__https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.TAO.html*petsc4py.PETSc.TAO.setGradientNorm__;Iw!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxNddlrzE$ ,
> this should do what I want. However, the method does not always seem to
> perform correctly: When I use it with “-tao_type lmvm”, it really seems to
> work and gives the correct scaling of the residual in the default TAO
> monitor. However, when I use, e.g., “-tao_type bqnls”, “-tao_type cg”, or
> “-tao_type bncg”, the (initial) residual is the same as it is when I do not
> use the TAOSetGradientNorm. However, there seem to be some slight internal
> changes (at least for the bqnls), as the number of iterations to reach the
> tolerance changes from 15 without TAOSetGradientNorm to 17 with
> TAOSetGradientNorm.
>
>
>
> For the context: Here, I am trying to solve a PDE constrained optimal
> control problem, which I tackle in a reduced fashion (using a reduced cost
> functional which results in an unconstrained optimization using the adjoint
> approach). For this, I would like to use the L2 inner product induced by
> the FEM discretization, so the L2 mass matrix.
>
>
>
> Moreover, I noticed that the performance of “-tao_type lmvm” and
> “-tao_type bqnls” as well as “-tao_type cg” and “-tao_type bncg” are
> drastically different for the same unconstrained problem. I would have
> expected that the algorithms are (more or less) identical for that case. Is
> this to be expected?
>
>
>
> Finally, I would like to use TAO for solving PDE constrained shape
> optimization problems. To do so, I would need to be able to specify the
> inner product used in the solver (see the above part) and this inner
> product would need to change in each iteration. Is it possible to do this
> with TAO? And could anyone give me some hints how to do so in python with
> petsc4py?
>
>
>
> Thanks a lot in advance,
>
> Sebastian
>
>
>
>
>
>
>
> --
>
> Dr. Sebastian Blauth
>
> Fraunhofer-Institut für
>
> Techno- und Wirtschaftsmathematik ITWM
>
> Abteilung Transportvorgänge
>
> Fraunhofer-Platz 1, 67663 Kaiserslautern
>
> Telefon: +49 631 31600-4968
>
> sebastian.blauth at itwm.fraunhofer.de
>
> https://urldefense.us/v3/__https://www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxKWFVczr$ 
>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxCQ5b78V$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxFbGYEze$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240712/6e0ca14d/attachment-0001.html>