<head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBannery1xbtsp { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButtony1xbtsp:hover, .pfptPrimaryButtony1xbtsp:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButtony1xbtsp:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
</head><!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
On Fri, Jul 12, 2024 at 7: 25 AM Blauth, Sebastian <sebastian. blauth@ itwm. fraunhofer. de> wrote: Dear Barry, thanks for the clarification. Oh, that’s unfortunate, that not all TAO algorithms use the supplied matrix for the norm (and then
</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBannery1xbtsp" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBannery1xbtsp" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBannery1xbtsp" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBannery1xbtsp" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<div dir="ltr"><div dir="ltr">On Fri, Jul 12, 2024 at 7:25 AM Blauth, Sebastian <<a href="mailto:sebastian.blauth@itwm.fraunhofer.de">sebastian.blauth@itwm.fraunhofer.de</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg3310274442017227626"><div lang="DE" style="overflow-wrap: break-word;"><div class="m_3310274442017227626WordSection1"><p class="MsoNormal"><span>Dear Barry,<u></u><u></u></span></p><p class="MsoNormal"><span><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US">thanks for the clarification. Oh, that’s unfortunate, that not all TAO algorithms use the supplied matrix for the norm (and then probably also not for computing inner products in, e.g., the limited memory formulas).<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US">I fear that I don’t have sufficient time at the moment to make a MR. I could, however, provide some “minimal” example where the behavior is shown. However, that example would be using petsc4py as I am only familiar with that and I would use the fenics FEM package to define the matrices. Would this be okay? </span></p></div></div></div></blockquote><div><br></div><div>Yes</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg3310274442017227626"><div lang="DE" style="overflow-wrap: break-word;"><div class="m_3310274442017227626WordSection1"><p class="MsoNormal"><span lang="EN-US">And if that’s the case, should I post the example here or at the petsc gitlab?</span></p></div></div></div></blockquote><div><br></div><div>Either place is fine. Gitlab makes it easier to track.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg3310274442017227626"><div lang="DE" style="overflow-wrap: break-word;"><div class="m_3310274442017227626WordSection1"><p class="MsoNormal"><span lang="EN-US">Best regards,<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US">Sebastian<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p><div><p class="MsoNormal">--<u></u><u></u></p><p class="MsoNormal">Dr. Sebastian Blauth<u></u><u></u></p><p class="MsoNormal">Fraunhofer-Institut für<u></u><u></u></p><p class="MsoNormal">Techno- und Wirtschaftsmathematik ITWM<u></u><u></u></p><p class="MsoNormal">Abteilung Transportvorgänge<u></u><u></u></p><p class="MsoNormal">Fraunhofer-Platz 1, 67663 Kaiserslautern<u></u><u></u></p><p class="MsoNormal">Telefon: +49 631 31600-4968<u></u><u></u></p><p class="MsoNormal"><a href="mailto:sebastian.blauth@itwm.fraunhofer.de" target="_blank">sebastian.blauth@itwm.fraunhofer.de</a><u></u><u></u></p><p class="MsoNormal"><a href="https://urldefense.us/v3/__https://www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxKWFVczr$" target="_blank">https://www.itwm.fraunhofer.de</a><u></u><u></u></p></div><p class="MsoNormal"><span><u></u> <u></u></span></p><div style="border-top:none;border-right:none;border-bottom:none;border-left:1.5pt solid blue;padding:0cm 0cm 0cm 4pt"><div><div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm 0cm"><p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> <br><b>Sent:</b> Tuesday, July 9, 2024 3:32 PM<br><b>To:</b> Blauth, Sebastian <<a href="mailto:sebastian.blauth@itwm.fraunhofer.de" target="_blank">sebastian.blauth@itwm.fraunhofer.de</a>>; Munson, Todd <<a href="mailto:tmunson@mcs.anl.gov" target="_blank">tmunson@mcs.anl.gov</a>>; toby Isaac <<a href="mailto:toby.isaac@gmail.com" target="_blank">toby.isaac@gmail.com</a>><br><b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br><b>Subject:</b> Re: [petsc-users] Questions on TAO and gradient norm / inner products<u></u><u></u></span></p></div></div><p class="MsoNormal"><u></u> <u></u></p><div><p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal"> From <u></u><u></u></p><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif">$ git grep TaoGradientNorm <u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">bound/impls/blmvm/blmvm.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">bound/impls/blmvm/blmvm.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">bound/impls/bnk/bnk.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &bnk->gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">bound/impls/bnk/bnk.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &bnk->gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">bound/impls/bnk/bnls.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &bnk->gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">bound/impls/bnk/bntl.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &bnk->gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">bound/impls/bnk/bntl.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &bnk->gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">bound/impls/bnk/bntr.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &bnk->gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">interface/taosolver.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif">.seealso: [](ch_tao), `Tao`, `TaoGetGradientNorm()`, `<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>()`<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">interface/taosolver.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif">.seealso: [](ch_tao), `Tao`, `TaoSetGradientNorm()`, `<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>()`<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">interface/taosolver.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> <b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b> - Compute the norm using the `NormType`, the user has selected<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">interface/taosolver.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif">PetscErrorCode <b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(Tao tao, Vec gradient, NormType type, PetscReal *gnorm)<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">unconstrained/impls/lmvm/lmvm.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">unconstrained/impls/lmvm/lmvm.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">unconstrained/impls/nls/nls.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">unconstrained/impls/nls/nls.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">unconstrained/impls/nls/nls.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">unconstrained/impls/ntr/ntr.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">unconstrained/impls/ntr/ntr.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(200,20,201)">unconstrained/impls/ntr/ntr.c</span><span style="font-size:10.5pt;font-family:Menlo,serif;color:rgb(46,174,187)">:</span><span style="font-size:10.5pt;font-family:Menlo,serif"> PetscCall(<b><span style="color:rgb(180,36,25)">TaoGradientNorm</span></b>(tao, tao->gradient, NORM_2, &gnorm));<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"><br><br><u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif">it appears only some of the algorithm implementations use the norm you provide. While <u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"><br><br><u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"> git grep VecNorm<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"><br><br><u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif">indicates many places where it is not used. Likely some of the other algorithm implementations could be easily "fixed" to support by<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif">changing the norm computed. But I am not an expert on the algorithms and don't know if all algorithms can mathematically support a user provided norm.<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"><u></u> <u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif">You are welcome to take a stab at making the change in an MR, or do you have a simple test problem with a mass matrix we can use to fix the<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif">"missing" implementations?<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"><u></u> <u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"> Barry<u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"><br><br><u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"><br><br><u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"><br><br><u></u><u></u></span></p><p style="margin:0cm;font-variant-caps:normal;font-stretch:normal;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal"><span style="font-size:10.5pt;font-family:Menlo,serif"><br><br><u></u><u></u></span></p><div><p class="MsoNormal"><br><br><u></u><u></u></p><blockquote style="margin-top:5pt;margin-bottom:5pt"><div><p class="MsoNormal">On Jul 9, 2024, at 3:47 AM, Blauth, Sebastian <<a href="mailto:sebastian.blauth@itwm.fraunhofer.de" target="_blank">sebastian.blauth@itwm.fraunhofer.de</a>> wrote:<u></u><u></u></p></div><p class="MsoNormal"><u></u> <u></u></p><div><div><p class="MsoNormal">Hello,<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US">I have some questions regarding TAO and the use the gradient norm.<span class="m_3310274442017227626apple-converted-space"> </span></span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US">First, I want to use a custom inner product for the optimization in TAO (for computing the gradient norm and, e.g., in the double loop of a quasi-Newton method). I have seen that there is the method TAOSetGradientNorm<span class="m_3310274442017227626apple-converted-space"> </span><a href="https://urldefense.us/v3/__https://petsc.org/release/manualpages/Tao/TaoSetGradientNorm/__;!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxDTBKkKq$" target="_blank"><span style="color:rgb(5,99,193)">https://petsc.org/release/manualpages/Tao/TaoSetGradientNorm/</span></a><span class="m_3310274442017227626apple-converted-space"> </span>which seems to do this. According to the petsc4py docs<span class="m_3310274442017227626apple-converted-space"> </span><a href="https://urldefense.us/v3/__https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.TAO.html*petsc4py.PETSc.TAO.setGradientNorm__;Iw!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxNddlrzE$" target="_blank"><span style="color:rgb(5,99,193)">https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.TAO.html#petsc4py.PETSc.TAO.setGradientNorm</span></a>, this should do what I want. However, the method does not always seem to perform correctly: When I use it with “-tao_type lmvm”, it really seems to work and gives the correct scaling of the residual in the default TAO monitor. However, when I use, e.g., “-tao_type bqnls”, “-tao_type cg”, or “-tao_type bncg”, the (initial) residual is the same as it is when I do not use the TAOSetGradientNorm. However, there seem to be some slight internal changes (at least for the bqnls), as the number of iterations to reach the tolerance changes from 15 without TAOSetGradientNorm to 17 with TAOSetGradientNorm.<span class="m_3310274442017227626apple-converted-space"> </span></span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US">For the context: Here, I am trying to solve a PDE constrained optimal control problem, which I tackle in a reduced fashion (using a reduced cost functional which results in an unconstrained optimization using the adjoint approach). For this, I would like to use the L2 inner product induced by the FEM discretization, so the L2 mass matrix.</span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US">Moreover, I noticed that the performance of “-tao_type lmvm” and “-tao_type bqnls” as well as “-tao_type cg” and “-tao_type bncg” are drastically different for the same unconstrained problem. I would have expected that the algorithms are (more or less) identical for that case. Is this to be expected?</span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US">Finally, I would like to use TAO for solving PDE constrained shape optimization problems. To do so, I would need to be able to specify the inner product used in the solver (see the above part) and this inner product would need to change in each iteration. Is it possible to do this with TAO? And could anyone give me some hints how to do so in python with petsc4py?</span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US">Thanks a lot in advance,</span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US">Sebastian</span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p></div><div><p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p></div><div><p class="MsoNormal">--<u></u><u></u></p></div><div><p class="MsoNormal">Dr. Sebastian Blauth<u></u><u></u></p></div><div><p class="MsoNormal">Fraunhofer-Institut für<u></u><u></u></p></div><div><p class="MsoNormal">Techno- und Wirtschaftsmathematik ITWM<u></u><u></u></p></div><div><p class="MsoNormal">Abteilung Transportvorgänge<u></u><u></u></p></div><div><p class="MsoNormal">Fraunhofer-Platz 1, 67663 Kaiserslautern<u></u><u></u></p></div><div><p class="MsoNormal">Telefon: +49 631 31600-4968<u></u><u></u></p></div><div><p class="MsoNormal"><a href="mailto:sebastian.blauth@itwm.fraunhofer.de" target="_blank">sebastian.blauth@itwm.fraunhofer.de</a><u></u><u></u></p></div><div><p class="MsoNormal"><a href="https://urldefense.us/v3/__https://www.itwm.fraunhofer.de/__;!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxPY3UEoZ$" target="_blank">https://www.itwm.fraunhofer.de</a><u></u><u></u></p></div></div></blockquote></div><p class="MsoNormal"><u></u> <u></u></p></div></div></div></div></div></blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!aSNMLW5E3ZNlZwOAfQJMnwBM_4sDBJ8jXhjmfgZFQJJJFSw-7QaqXFgtPRpKudragLDd8ONBV5pJxFbGYEze$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>