<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
In Sajid’s problem, the optimization variables (F_t in the equation u_t = A*(u_xx + u_yy) + F_t*u) are complex-valued. The gradients should also be complex-valued. The objective function may be real-valued.
<div class=""><br class="">
</div>
<div class="">Hong (Mr.)<br class="">
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Apr 14, 2020, at 5:52 PM, Dener, Alp <<a href="mailto:adener@anl.gov" class="">adener@anl.gov</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="auto" class="">This is correct. As long as the optimization variables and the objective function, and it’s gradient are real valued, intermediate variables (such as PDE states) can be complex.
<div class=""><br class="">
</div>
<div class="">In principle it is also possible to minimize real valued functions in complex variables by converting to rectangular or polar coordinate space and working with real numbers only (this will double the size of the optimization problem). The same
transformation for complex valued functions yields a multi objective optimization problem. </div>
<div class=""><br class="">
</div>
<div class="">There’s no guarantee though that TAO algorithms will work with this out of the box. When PETSc is compiled complex, the above transformation yields TAO solution vectors that are still complex but carry all the information in the real component
with zeros in the imaginary. Linear algebra with these vectors may not turn out to be equivalent to the real-compiled counterparts. </div>
<div class=""><br class="">
</div>
<div class="">This potential issue also applies to carrying around complex state variables in a PDE constrained problem. Even though the optimization algorithm never sees them, the PETSc data structures in TAO would still be complex valued with zeros in the
imaginary. We’ve never tested TAO this way. </div>
<div class=""><br class="">
</div>
<div class="">Alp</div>
<div class=""><br class="">
</div>
<div class="">On Apr 14, 2020, at 5:26 PM, Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" class="">stefano.zampini@gmail.com</a>> wrote:</div>
<div class="">
<div dir="ltr" class="">
<blockquote type="cite" class=""><br class="">
</blockquote>
</div>
<blockquote type="cite" class="">
<div dir="ltr" class=""> Not true in general when you minimize an objective function as a functional of the parameter only
<div class="">For same methods (Newton for example, gradient descent, etc) the state variables do no enter the minimization, so it should be fine to have complex-valued state variables</div>
<div class=""><br class="">
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On Apr 15, 2020, at 1:04 AM, Zhang, Hong via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" class="">petsc-users@mcs.anl.gov</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Sorry for the time travel. As far as I know, optimization over complex-valued parameters is not a well-defined problem. I am not sure how you can develop an optimization algorithm for it. Perhaps our optimization experts have better suggestions in this direction.
<div class=""><br class="">
</div>
<div class="">The real-valued formulation seems to be more promising to me. The preconditioning is hard, but still doable with fieldsplit as Mark mentioned.</div>
<div class=""><br class="">
</div>
<div class="">Hong (Mr.)<br class="">
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On Apr 14, 2020, at 1:42 PM, Sajid Ali <<a href="mailto:sajidsyed2021@u.northwestern.edu" class="">sajidsyed2021@u.northwestern.edu</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class="">
<div class="">Hi Hong, <br class="">
<br class="">
</div>
Apologies for creating unnecessary confusion by continuing the old thread instead of creating a new one.
<br class="">
<br class="">
</div>
<div class="">While I looked into converting the complex PDE formulation to a real valued formulation in the past hoping for better performance, my concern now is with TAO being incompatible with complex scalars. I would've preferred to keep the complex PDE
formulation as is (given that I spent some time tuning it and it works well now) for cost function and gradient evaluation while using TAO for the outer optimization loop.
<br class="">
<br class="">
</div>
<div class="">Using TAO has the obvious benefit of defining a multi objective cost function, parametrized as a fit to a series of measurements and a set of regularizers while not having to explicitly worry about differentiating the regularizer or have to think
about implementing a good optimization scheme. But if it converting the complex formulation to a real formulation would mean a loss of well conditioned forward solve (and increase in solving time itself), I was wondering if it would be better to keep the complex
PDE formulation and write an optimization loop in PETSc while defining the regularizer via a cost integrand.<br class="">
</div>
<br class="">
</div>
Thank You, <br class="">
<div class="">
<div dir="ltr" class="gmail_signature">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class="">
<div style="font-size:12.8px" class="">Sajid Ali | PhD Candidate<br class="">
</div>
<div style="font-size:12.8px" class="">Applied Physics<br class="">
</div>
<div style="font-size:12.8px" class="">Northwestern University</div>
<div style="font-size:12.8px" class=""><a href="http://s-sajid-ali.github.io/" target="_blank" class="">s-sajid-ali.github.io</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</body>
</html>