[petsc-users] Questions on TAO and gradient norm / inner products

[Barry Smith]

   From 

$ git grep TaoGradientNorm 
bound/impls/blmvm/blmvm.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));
bound/impls/blmvm/blmvm.c:    PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));
bound/impls/bnk/bnk.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));
bound/impls/bnk/bnk.c:          PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));
bound/impls/bnk/bnls.c:      PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));
bound/impls/bnk/bntl.c:      PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));
bound/impls/bnk/bntl.c:        PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));
bound/impls/bnk/bntr.c:        PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));
interface/taosolver.c:.seealso: [](ch_tao), `Tao`, `TaoGetGradientNorm()`, `TaoGradientNorm()`
interface/taosolver.c:.seealso: [](ch_tao), `Tao`, `TaoSetGradientNorm()`, `TaoGradientNorm()`
interface/taosolver.c:  TaoGradientNorm - Compute the norm using the `NormType`, the user has selected
interface/taosolver.c:PetscErrorCode TaoGradientNorm(Tao tao, Vec gradient, NormType type, PetscReal *gnorm)
unconstrained/impls/lmvm/lmvm.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));
unconstrained/impls/lmvm/lmvm.c:      PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));
unconstrained/impls/nls/nls.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));
unconstrained/impls/nls/nls.c:          PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));
unconstrained/impls/nls/nls.c:    PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));
unconstrained/impls/ntr/ntr.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));
unconstrained/impls/ntr/ntr.c:        PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));
unconstrained/impls/ntr/ntr.c:      PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

it appears only some of the algorithm implementations use the norm you provide. While 

 git grep VecNorm

indicates many places where it is not used. Likely some of the other algorithm implementations could be easily "fixed" to support by
changing the norm computed. But I am not an expert on the algorithms and don't know if all algorithms can mathematically support a user provided norm.

You are welcome to take a stab at making the change in an MR, or do you have a simple test problem with a mass matrix we can use to fix the
"missing" implementations?

  Barry





> On Jul 9, 2024, at 3:47 AM, Blauth, Sebastian <sebastian.blauth at itwm.fraunhofer.de> wrote:
> 
> Hello,
>  
> I have some questions regarding TAO and the use the gradient norm. 
>  
> First, I want to use a custom inner product for the optimization in TAO (for computing the gradient norm and, e.g., in the double loop of a quasi-Newton method). I have seen that there is the method TAOSetGradientNorm https://urldefense.us/v3/__https://petsc.org/release/manualpages/Tao/TaoSetGradientNorm/__;!!G_uCfscf7eWS!YYY6wdyL1EFFToD3UIx3H-GCsAk3BjMjp0gbO1zDgnDDViycNJ0OgClf3PxjdA7J-xzTSOrACClqwdPuNgYUsmo$  which seems to do this. According to the petsc4py docs https://urldefense.us/v3/__https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.TAO.html*petsc4py.PETSc.TAO.setGradientNorm__;Iw!!G_uCfscf7eWS!YYY6wdyL1EFFToD3UIx3H-GCsAk3BjMjp0gbO1zDgnDDViycNJ0OgClf3PxjdA7J-xzTSOrACClqwdPutG_lawA$ , this should do what I want. However, the method does not always seem to perform correctly: When I use it with “-tao_type lmvm”, it really seems to work and gives the correct scaling of the residual in the default TAO monitor. However, when I use, e.g., “-tao_type bqnls”, “-tao_type cg”, or “-tao_type bncg”, the (initial) residual is the same as it is when I do not use the TAOSetGradientNorm. However, there seem to be some slight internal changes (at least for the bqnls), as the number of iterations to reach the tolerance changes from 15 without TAOSetGradientNorm to 17 with TAOSetGradientNorm. 
>  
> For the context: Here, I am trying to solve a PDE constrained optimal control problem, which I tackle in a reduced fashion (using a reduced cost functional which results in an unconstrained optimization using the adjoint approach). For this, I would like to use the L2 inner product induced by the FEM discretization, so the L2 mass matrix.
>  
> Moreover, I noticed that the performance of “-tao_type lmvm” and “-tao_type bqnls” as well as “-tao_type cg” and “-tao_type bncg” are drastically different for the same unconstrained problem. I would have expected that the algorithms are (more or less) identical for that case. Is this to be expected?
>  
> Finally, I would like to use TAO for solving PDE constrained shape optimization problems. To do so, I would need to be able to specify the inner product used in the solver (see the above part) and this inner product would need to change in each iteration. Is it possible to do this with TAO? And could anyone give me some hints how to do so in python with petsc4py?
>  
> Thanks a lot in advance,
> Sebastian
>  
>  
>  
> --
> Dr. Sebastian Blauth
> Fraunhofer-Institut für
> Techno- und Wirtschaftsmathematik ITWM
> Abteilung Transportvorgänge
> Fraunhofer-Platz 1, 67663 Kaiserslautern
> Telefon: +49 631 31600-4968
> sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de>
> https://urldefense.us/v3/__https://www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!YYY6wdyL1EFFToD3UIx3H-GCsAk3BjMjp0gbO1zDgnDDViycNJ0OgClf3PxjdA7J-xzTSOrACClqwdPuQJOW8S8$  <https://urldefense.us/v3/__https://www.itwm.fraunhofer.de/__;!!G_uCfscf7eWS!YYY6wdyL1EFFToD3UIx3H-GCsAk3BjMjp0gbO1zDgnDDViycNJ0OgClf3PxjdA7J-xzTSOrACClqwdPuPeTzpvg$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240709/5d5e1bd3/attachment-0001.html>