<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div apple-content-edited="true" class="">Hey Jason:
</div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class="">The line search fails. If I use Armijo I get</div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class=""><div apple-content-edited="true" class="">TaoLineSearch Object:</div><div apple-content-edited="true" class=""> type: armijo</div><div apple-content-edited="true" class=""> maxf=30, ftol=1e-10, gtol=0.0001</div><div apple-content-edited="true" class=""> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 memsize=1</div><div apple-content-edited="true" class=""> maximum function evaluations=30</div><div apple-content-edited="true" class=""> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9</div><div apple-content-edited="true" class=""> total number of function evaluations=1</div><div apple-content-edited="true" class=""> total number of gradient evaluations=1</div><div apple-content-edited="true" class=""> total number of function/gradient evaluations=0</div><div apple-content-edited="true" class=""> Termination reason: 0</div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class="">The parameters seem to be the default ones also suggested by Nocedal and Wright. So I did not change anything. The termination reason is equivalent to TAOLINESEARCH_CONTINUE_ITERATING. I am not checking the reason directly. I guess it starts reducing the step size after that. I can see that my objective function get’s evaluated (as expected); however, the objective values increase (from what I see when monitoring the evaluations of my objective). This leads to a failure in the line search and made (still makes) me believe there is a bug on my side (which I have not found yet). However, if I use a unit step it converges (relative change of the gradient e.g. to 1E-9; see bottom of this email). If I use More & Thuente, same thing. No reduction in step size necessary.</div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class="">If you suggest that I should do some further testing on simpler problems, I’m happy to do so. After looking at the code, I just felt like there obviously is something wrong in the line-search implementation.</div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class="">Thanks for your help.</div><div apple-content-edited="true" class="">/Andreas</div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class=""><b class="">Here’s the output after the first iteration (where the Armijo line search fails):</b></div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class=""><div apple-content-edited="true" class="">TaoLineSearch Object:</div><div apple-content-edited="true" class=""> type: armijo</div><div apple-content-edited="true" class=""> maxf=30, ftol=1e-10, gtol=0.0001</div><div apple-content-edited="true" class=""> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 memsize=1</div><div apple-content-edited="true" class=""> maximum function evaluations=30</div><div apple-content-edited="true" class=""> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9</div><div apple-content-edited="true" class=""> total number of function evaluations=1</div><div apple-content-edited="true" class=""> total number of gradient evaluations=1</div><div apple-content-edited="true" class=""> total number of function/gradient evaluations=0</div><div apple-content-edited="true" class=""> Termination reason: 0</div><div apple-content-edited="true" class="">TaoLineSearch Object:</div><div apple-content-edited="true" class=""> type: armijo</div><div apple-content-edited="true" class=""> maxf=30, ftol=1e-10, gtol=0.0001</div><div apple-content-edited="true" class=""> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 memsize=1</div><div apple-content-edited="true" class=""> maximum function evaluations=30</div><div apple-content-edited="true" class=""> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9</div><div apple-content-edited="true" class=""> total number of function evaluations=30</div><div apple-content-edited="true" class=""> total number of gradient evaluations=0</div><div apple-content-edited="true" class=""> total number of function/gradient evaluations=0</div><div apple-content-edited="true" class=""> Termination reason: 4</div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class=""><b class="">With final output (end of optimization):</b></div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class=""><div apple-content-edited="true" class="">Tao Object: 1 MPI processes</div><div apple-content-edited="true" class=""> type: nls</div><div apple-content-edited="true" class=""> Newton steps: 1</div><div apple-content-edited="true" class=""> BFGS steps: 0</div><div apple-content-edited="true" class=""> Scaled gradient steps: 0</div><div apple-content-edited="true" class=""> Gradient steps: 1</div><div apple-content-edited="true" class=""> nls ksp atol: 0</div><div apple-content-edited="true" class=""> nls ksp rtol: 1</div><div apple-content-edited="true" class=""> nls ksp ctol: 0</div><div apple-content-edited="true" class=""> nls ksp negc: 0</div><div apple-content-edited="true" class=""> nls ksp dtol: 0</div><div apple-content-edited="true" class=""> nls ksp iter: 0</div><div apple-content-edited="true" class=""> nls ksp othr: 0</div><div apple-content-edited="true" class=""> TaoLineSearch Object: 1 MPI processes</div><div apple-content-edited="true" class=""> type: armijo</div><div apple-content-edited="true" class=""> KSP Object: 1 MPI processes</div><div apple-content-edited="true" class=""> type: cg</div><div apple-content-edited="true" class=""> total KSP iterations: 21</div><div apple-content-edited="true" class=""> convergence tolerances: fatol=0, frtol=0</div><div apple-content-edited="true" class=""> convergence tolerances: gatol=0, steptol=0, gttol=0.0001</div><div apple-content-edited="true" class=""> Residual in Function/Gradient:=0.038741</div><div apple-content-edited="true" class=""> Objective value=0.639121</div><div apple-content-edited="true" class=""> total number of iterations=0, (max: 50)</div><div apple-content-edited="true" class=""> total number of function evaluations=31, max: 10000</div><div apple-content-edited="true" class=""> total number of gradient evaluations=1, max: 10000</div><div apple-content-edited="true" class=""> total number of function/gradient evaluations=1, (max: 10000)</div><div apple-content-edited="true" class=""> total number of Hessian evaluations=1</div><div apple-content-edited="true" class=""> Solver terminated: -6 Line Search Failure</div></div></div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class=""><b class="">This is without line-search (unit step size):</b></div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class=""><div apple-content-edited="true" class="">Tao Object: 1 MPI processes</div><div apple-content-edited="true" class=""> type: nls</div><div apple-content-edited="true" class=""> Newton steps: 3</div><div apple-content-edited="true" class=""> BFGS steps: 0</div><div apple-content-edited="true" class=""> Scaled gradient steps: 0</div><div apple-content-edited="true" class=""> Gradient steps: 0</div><div apple-content-edited="true" class=""> nls ksp atol: 0</div><div apple-content-edited="true" class=""> nls ksp rtol: 3</div><div apple-content-edited="true" class=""> nls ksp ctol: 0</div><div apple-content-edited="true" class=""> nls ksp negc: 0</div><div apple-content-edited="true" class=""> nls ksp dtol: 0</div><div apple-content-edited="true" class=""> nls ksp iter: 0</div><div apple-content-edited="true" class=""> nls ksp othr: 0</div><div apple-content-edited="true" class=""> TaoLineSearch Object: 1 MPI processes</div><div apple-content-edited="true" class=""> type: unit</div><div apple-content-edited="true" class=""> KSP Object: 1 MPI processes</div><div apple-content-edited="true" class=""> type: cg</div><div apple-content-edited="true" class=""> total KSP iterations: 71</div><div apple-content-edited="true" class=""> convergence tolerances: fatol=0, frtol=0</div><div apple-content-edited="true" class=""> convergence tolerances: gatol=0, steptol=0, gttol=0.0001</div><div apple-content-edited="true" class=""> Residual in Function/Gradient:=1.91135e-11</div><div apple-content-edited="true" class=""> Objective value=0.160914</div><div apple-content-edited="true" class=""> total number of iterations=3, (max: 50)</div><div apple-content-edited="true" class=""> total number of function/gradient evaluations=4, (max: 10000)</div><div apple-content-edited="true" class=""> total number of Hessian evaluations=3</div><div apple-content-edited="true" class=""> Solution converged: ||g(X)||/||g(X0)|| <= gttol</div></div></div><div apple-content-edited="true" class=""><br class=""></div><div apple-content-edited="true" class=""> </div>
<br class=""><div class=""><blockquote type="cite" class=""><div class="">On Jun 11, 2015, at 12:44 PM, Jason Sarich <<a href="mailto:jason.sarich@gmail.com" class="">jason.sarich@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Hi Andreas,<div class=""><br class=""></div><div class="">Yes it looks like a bug that the reason is never set, but the line should still terminate. Is the problem you are having with the line search itself, or is it failing because you are checking this ls->reason directly?</div><div class=""><br class=""></div><div class="">Jason Sarich</div><div class=""><br class=""></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Thu, Jun 11, 2015 at 9:53 AM, Andreas Mang <span dir="ltr" class=""><<a href="mailto:andreas@ices.utexas.edu" target="_blank" class="">andreas@ices.utexas.edu</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi guys:<br class="">
<br class="">
I have a problem with the TAO Armijo line search (petsc-3.5.4). My algorithm works if I use the More & Thuente line search (default). I have numerically checked the gradient of my objective. It’s correct. I am happy to write a small snippet of code and do an easy test if you guys disagree, but from what I’ve seen in the line search code it seems obvious to me that there is a bug. Am I missing something or are you not setting<br class="">
<br class="">
ls->reason<br class="">
<br class="">
to<br class="">
<br class="">
TAOLINESEARCH_SUCCESS<br class="">
<br class="">
if the Armijo condition is fulfilled (TaoLineSearchApply_Armijo in armijo.c; line 118 - 302)?!<br class="">
<br class="">
It seems to me that ls->reason is and will remain to be set to<br class="">
<br class="">
TAOLINESEARCH_CONTINUE_ITERATING<br class="">
<br class="">
if everything works (i.e. I don’t hit one of the exceptions). Does this make sense? If not I’ll invest the time and put together a simple test case and, if that works, continue to check my code.<br class="">
<span class="HOEnZb"><font color="#888888" class=""><br class="">
/Andreas<br class="">
<br class="">
<br class="">
</font></span></blockquote></div><br class=""></div>
</div></blockquote></div><br class=""></body></html>