<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi Hansol,
<div class=""><br class="">
</div>
<div class="">We don’t have a Levenberg-Marquardt method available, and if the PETSc/TAO manual says otherwise, that may be misleading. Let me know where you saw that and I can take a look and fix it.</div>
<div class=""><br class="">
</div>
<div class="">In the meantime, if you want to solve a least-squares problem, the master branch of PETSc on Bitbucket has a bound-constrained regularized Gauss-Newton (TAOBRGN) method available. The only available regularization right now is an L2 proximal point
Tikhonov regularizer. There are ongoing efforts to support an L1 regularizer, and also the ability for users to define their own, but these have not made it into the master branch yet. We’re working on it and should be in for the next major PETSc release in
the Spring.</div>
<div class=""><br class="">
</div>
<div class="">If you’d like to use that method, you need to set the Tao type to TAOBRGN and then go through the TaoSetResidualRoutine() and TaoSetJacobianResidualRoutine() interfaces to define your problem.</div>
<div class=""><br class="">
</div>
<div class="">In general, you can use other TAO algorithms (e.g.: BNLS, BQNLS, etc.) with your own regularization term by embedding it into the objective, gradient and Hessian (if applicable) evaluation callbacks. The caveat is that your regularizer needs to
be C1 continuous for first-order methods and C2 continuous for second order methods. This typically limits you to L2-norm regularizers. There is no support yet for L1-norm regularizers, but as I said, we’re working on it right now and it should be available
in a couple of months.</div>
<div class=""><br class="">
</div>
<div class="">Hope that helps,</div>
<div class="">
<div class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
——<br class="">
<b class="">Alp Dener</b></div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Argonne National Laboratory</div>
<div style="text-align: start; text-indent: 0px;"><a href="https://www.anl.gov/profile/alp-dener" class="">https://www.anl.gov/profile/alp-dener</a></div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br class="">
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Jan 24, 2019, at 2:57 PM, David via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" class="">petsc-users@mcs.anl.gov</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">Hi. I was wondering whether there was some kind of general consensus about<br class="">
<br class="">
the currently-best-implemented L1, L2 norm regularization for petsc/tao <br class="">
that has been implemented.<br class="">
<br class="">
Naively, I would shoot for Levenberg-Marquardt for some kind of random <br class="">
matrix, or even generic<br class="">
<br class="">
finite-difference stencil problem. (but it seems like LM is yet to be <br class="">
implemented, but only on petsc manual pdf?)<br class="">
<br class="">
Or perhaps, of the implemented ones, LMVM seems to work well, at least <br class="">
on my local machine.<br class="">
<br class="">
In any due case, I would highly appreciate the input and opinion about <br class="">
these matters.<br class="">
<br class="">
<br class="">
Thanks.<br class="">
<br class="">
<br class="">
<br class="">
Hansol Suh,<br class="">
<br class="">
PhD Student<br class="">
<br class="">
<br class="">
Georgia Institute of Technology<br class="">
<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</body>
</html>