[petsc-users] Questions on TAO and gradient norm / inner products

[Blauth, Sebastian]

Dear Barry,

 

thanks for the clarification. Oh, that’s unfortunate, that not all TAO algorithms use the supplied matrix for the norm (and then probably also not for computing inner products in, e.g., the limited memory formulas).

 

I fear that I don’t have sufficient time at the moment to make a MR. I could, however, provide some “minimal” example where the behavior is shown. However, that example would be using petsc4py as I am only familiar with that and I would use the fenics FEM package to define the matrices. Would this be okay? And if that’s the case, should I post the example here or at the petsc gitlab?

 

Best regards,

Sebastian

 

--

Dr. Sebastian Blauth

Fraunhofer-Institut für

Techno- und Wirtschaftsmathematik ITWM

Abteilung Transportvorgänge

Fraunhofer-Platz 1, 67663 Kaiserslautern

Telefon: +49 631 31600-4968

sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de> 

https://www.itwm.fraunhofer.de

 

From: Barry Smith <bsmith at petsc.dev> 
Sent: Tuesday, July 9, 2024 3:32 PM
To: Blauth, Sebastian <sebastian.blauth at itwm.fraunhofer.de>; Munson, Todd <tmunson at mcs.anl.gov>; toby Isaac <toby.isaac at gmail.com>
Cc: petsc-users at mcs.anl.gov
Subject: Re: [petsc-users] Questions on TAO and gradient norm / inner products

 

 

   From 

 

$ git grep TaoGradientNorm 

bound/impls/blmvm/blmvm.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

bound/impls/blmvm/blmvm.c:    PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

bound/impls/bnk/bnk.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));

bound/impls/bnk/bnk.c:          PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));

bound/impls/bnk/bnls.c:      PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));

bound/impls/bnk/bntl.c:      PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));

bound/impls/bnk/bntl.c:        PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));

bound/impls/bnk/bntr.c:        PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &bnk->gnorm));

interface/taosolver.c:.seealso: [](ch_tao), `Tao`, `TaoGetGradientNorm()`, `TaoGradientNorm()`

interface/taosolver.c:.seealso: [](ch_tao), `Tao`, `TaoSetGradientNorm()`, `TaoGradientNorm()`

interface/taosolver.c:  TaoGradientNorm - Compute the norm using the `NormType`, the user has selected

interface/taosolver.c:PetscErrorCode TaoGradientNorm(Tao tao, Vec gradient, NormType type, PetscReal *gnorm)

unconstrained/impls/lmvm/lmvm.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

unconstrained/impls/lmvm/lmvm.c:      PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

unconstrained/impls/nls/nls.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

unconstrained/impls/nls/nls.c:          PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

unconstrained/impls/nls/nls.c:    PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

unconstrained/impls/ntr/ntr.c:  PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

unconstrained/impls/ntr/ntr.c:        PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));

unconstrained/impls/ntr/ntr.c:      PetscCall(TaoGradientNorm(tao, tao->gradient, NORM_2, &gnorm));





it appears only some of the algorithm implementations use the norm you provide. While 





 git grep VecNorm





indicates many places where it is not used. Likely some of the other algorithm implementations could be easily "fixed" to support by

changing the norm computed. But I am not an expert on the algorithms and don't know if all algorithms can mathematically support a user provided norm.

 

You are welcome to take a stab at making the change in an MR, or do you have a simple test problem with a mass matrix we can use to fix the

"missing" implementations?

 

  Barry





















On Jul 9, 2024, at 3:47 AM, Blauth, Sebastian <sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de> > wrote:

 

Hello,

 

I have some questions regarding TAO and the use the gradient norm. 

 

First, I want to use a custom inner product for the optimization in TAO (for computing the gradient norm and, e.g., in the double loop of a quasi-Newton method). I have seen that there is the method TAOSetGradientNorm  <https://petsc.org/release/manualpages/Tao/TaoSetGradientNorm/> https://petsc.org/release/manualpages/Tao/TaoSetGradientNorm/ which seems to do this. According to the petsc4py docs  <https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.TAO.html#petsc4py.PETSc.TAO.setGradientNorm> https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.TAO.html#petsc4py.PETSc.TAO.setGradientNorm, this should do what I want. However, the method does not always seem to perform correctly: When I use it with “-tao_type lmvm”, it really seems to work and gives the correct scaling of the residual in the default TAO monitor. However, when I use, e.g., “-tao_type bqnls”, “-tao_type cg”, or “-tao_type bncg”, the (initial) residual is the same as it is when I do not use the TAOSetGradientNorm. However, there seem to be some slight internal changes (at least for the bqnls), as the number of iterations to reach the tolerance changes from 15 without TAOSetGradientNorm to 17 with TAOSetGradientNorm. 

 

For the context: Here, I am trying to solve a PDE constrained optimal control problem, which I tackle in a reduced fashion (using a reduced cost functional which results in an unconstrained optimization using the adjoint approach). For this, I would like to use the L2 inner product induced by the FEM discretization, so the L2 mass matrix.

 

Moreover, I noticed that the performance of “-tao_type lmvm” and “-tao_type bqnls” as well as “-tao_type cg” and “-tao_type bncg” are drastically different for the same unconstrained problem. I would have expected that the algorithms are (more or less) identical for that case. Is this to be expected?

 

Finally, I would like to use TAO for solving PDE constrained shape optimization problems. To do so, I would need to be able to specify the inner product used in the solver (see the above part) and this inner product would need to change in each iteration. Is it possible to do this with TAO? And could anyone give me some hints how to do so in python with petsc4py?

 

Thanks a lot in advance,

Sebastian

 

 

 

--

Dr. Sebastian Blauth

Fraunhofer-Institut für

Techno- und Wirtschaftsmathematik ITWM

Abteilung Transportvorgänge

Fraunhofer-Platz 1, 67663 Kaiserslautern

Telefon: +49 631 31600-4968

sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de> 

https://www.itwm.fraunhofer.de <https://www.itwm.fraunhofer.de/> 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240712/71de5599/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 7943 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240712/71de5599/attachment-0001.p7s>