<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hey Jason:<div class=""><br class=""></div><div class="">I’ve added the flag and it works like a charm (see output below). Yaaayyyy! :)</div><div class=""><br class=""></div><div class="">Thanks! /A</div><div class=""><br class=""></div><div class="">P.S: I still wonder why the successive evaluations of the objective kicked me out, though (i.e. why I did get monotonically *increasing* objective values; I also tried to switch the search direction using “tao_ls_armijo_nondescending” which did not help). Anyway...</div><div class=""><br class=""></div><div class="">Output:</div><div class=""><br class=""></div><div class=""><div class="">Tao Object: 1 MPI processes</div><div class=""> type: nls</div><div class=""> Newton steps: 2</div><div class=""> BFGS steps: 0</div><div class=""> Scaled gradient steps: 0</div><div class=""> Gradient steps: 0</div><div class=""> nls ksp atol: 0</div><div class=""> nls ksp rtol: 2</div><div class=""> nls ksp ctol: 0</div><div class=""> nls ksp negc: 0</div><div class=""> nls ksp dtol: 0</div><div class=""> nls ksp iter: 0</div><div class=""> nls ksp othr: 0</div><div class=""> TaoLineSearch Object: 1 MPI processes</div><div class=""> type: armijo</div><div class=""> KSP Object: 1 MPI processes</div><div class=""> type: cg</div><div class=""> total KSP iterations: 31</div><div class=""> convergence tolerances: fatol=0, frtol=0</div><div class=""> convergence tolerances: gatol=0, steptol=0, gttol=0.0001</div><div class=""> Residual in Function/Gradient:=2.69882e-06</div><div class=""> Objective value=0.370287</div><div class=""> total number of iterations=2, (max: 50)</div><div class=""> total number of function evaluations=2, max: 10000</div><div class=""> total number of gradient evaluations=2, max: 10000</div><div class=""> total number of function/gradient evaluations=1, (max: 10000)</div><div class=""> total number of Hessian evaluations=2</div><div class=""> Solution converged: ||g(X)||/||g(X0)|| <= gttol</div></div><div class=""><br class=""></div><div class=""><div apple-content-edited="true" class=""><br class=""></div><div><blockquote type="cite" class=""><div class="">On Jun 11, 2015, at 4:13 PM, Jason Sarich <<a href="mailto:jason.sarich@gmail.com" class="">jason.sarich@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Hi Andreas, <div class=""><br class=""></div><div class="">I'm pretty sure this is a bug on my side, directly related to the ls->reason<br class=""></div><div class=""><br class=""></div><div class="">Can you try fixing src/tao/linesearch/impls/armijo/armijo.c so it sets this correctly? </div><div class=""><br class=""></div><div class="">before (line 275):</div><div class=""><div class=""> /* Successful termination, update memory */</div><div class=""> armP->lastReference = ref;</div></div><div class=""><br class=""></div><div class=""><div class=""> /* Successful termination, update memory */</div><div class=""> ls->reason = TAOLINESEARCH_SUCCESS;</div><div class=""> armP->lastReference = ref;</div></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Thu, Jun 11, 2015 at 3:26 PM, Andreas Mang <span dir="ltr" class=""><<a href="mailto:andreas@ices.utexas.edu" target="_blank" class="">andreas@ices.utexas.edu</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word" class="">
Hey Jason:
<div class=""><br class="">
</div>
<div class=""><span class="">
<div class="">Thanks for looking into this. In the meantime I have checked against your elliptic tao example using an Armijo linesearch. It works (i.e. converges) as you suggested in an earlier email even though it returns the “wrong" flag. For my problem it
does even choke if I set the regularization parameter to 1E6 (essentially solving a quadratic problem).</div>
<div class=""><br class="">
</div>
<div class="">A final question before I continue the struggle by myself: It says “gradient steps: 1” instead of “gradient steps: 0” in the outputs (Armijo vs. More-Thuente / Unit). Does it start doing gradient evaluations? Maybe this helps me to further poke
my code. </div>
<div class=""><br class="">
</div>
<div class="">I’ll continue to look into this. I’ll come back to you if I discover that the problem is on the PETSc side of things and I can reproduce the problem with a toy example.</div>
<div class=""><br class="">
</div>
<div class="">Thanks for your time! /Andreas</div>
<div class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class="">
<span style="border-collapse:separate;border-spacing:0px" class=""><br class="">
</span></div>
</div>
</div>
<br class="">
</span><div class=""><div class="h5"><div class="">
<blockquote type="cite" class="">
<div class="">On Jun 11, 2015, at 3:03 PM, Jason Sarich <<a href="mailto:jason.sarich@gmail.com" target="_blank" class="">jason.sarich@gmail.com</a>> wrote:</div>
<br class="">
<div class="">
<div dir="ltr" class="">Hi Andreas,
<div class=""><br class="">
</div>
<div class="">I don't see anything obviously wrong. If the function is very flat, you can try setting -tao_ls_armijo_sigma to a smaller number. If you continue to have problems, please let me know. It would definitely help if you have an example you could send
me that reproduces this behavior.</div>
<div class=""><br class="">
</div>
<div class="">Jason</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">On Thu, Jun 11, 2015 at 1:12 PM, Andreas Mang <span dir="ltr" class="">
<<a href="mailto:andreas@ices.utexas.edu" target="_blank" class="">andreas@ices.utexas.edu</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word" class="">
<div class="">
<div class="">
<div class="">Hey Jason: </div>
<div class=""><br class="">
</div>
<div class="">The line search fails. If I use Armijo I get</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">TaoLineSearch Object:</div>
<div class=""> type: armijo</div>
<div class=""> maxf=30, ftol=1e-10, gtol=0.0001</div>
<div class=""> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 memsize=1</div>
<div class=""> maximum function evaluations=30</div>
<div class=""> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9</div>
<div class=""> total number of function evaluations=1</div>
<div class=""> total number of gradient evaluations=1</div>
<div class=""> total number of function/gradient evaluations=0</div>
<div class=""> Termination reason: 0</div>
<div class=""><br class="">
</div>
<div class="">The parameters seem to be the default ones also suggested by Nocedal and Wright. So I did not change anything. The termination reason is equivalent to TAOLINESEARCH_CONTINUE_ITERATING. I am not checking the reason directly. I guess it starts reducing
the step size after that. I can see that my objective function get’s evaluated (as expected); however, the objective values increase (from what I see when monitoring the evaluations of my objective). This leads to a failure in the line search and made (still
makes) me believe there is a bug on my side (which I have not found yet). However, if I use a unit step it converges (relative change of the gradient e.g. to 1E-9; see bottom of this email). If I use More & Thuente, same thing. No reduction in step size necessary.</div>
<div class=""><br class="">
</div>
<div class="">If you suggest that I should do some further testing on simpler problems, I’m happy to do so. After looking at the code, I just felt like there obviously is something wrong in the line-search implementation.</div>
<div class=""><br class="">
</div>
<div class="">Thanks for your help.</div>
<div class="">/Andreas</div>
<div class=""><br class="">
</div>
<div class=""><b class="">Here’s the output after the first iteration (where the Armijo line search fails):</b></div>
<div class=""><br class="">
</div>
<div class="">
<div class="">TaoLineSearch Object:</div>
<div class=""> type: armijo</div>
<div class=""> maxf=30, ftol=1e-10, gtol=0.0001</div>
<div class=""> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 memsize=1</div>
<div class=""> maximum function evaluations=30</div>
<div class=""> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9</div>
<div class=""> total number of function evaluations=1</div>
<div class=""> total number of gradient evaluations=1</div>
<div class=""> total number of function/gradient evaluations=0</div>
<div class=""> Termination reason: 0</div>
<div class="">TaoLineSearch Object:</div>
<div class=""> type: armijo</div>
<div class=""> maxf=30, ftol=1e-10, gtol=0.0001</div>
<div class=""> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001 memsize=1</div>
<div class=""> maximum function evaluations=30</div>
<div class=""> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9</div>
<div class=""> total number of function evaluations=30</div>
<div class=""> total number of gradient evaluations=0</div>
<div class=""> total number of function/gradient evaluations=0</div>
<div class=""> Termination reason: 4</div>
<div class=""><br class="">
</div>
<div class=""><b class="">With final output (end of optimization):</b></div>
<div class=""><br class="">
</div>
<div class="">
<div class="">Tao Object: 1 MPI processes</div>
<div class=""> type: nls</div>
<div class=""> Newton steps: 1</div>
<div class=""> BFGS steps: 0</div>
<div class=""> Scaled gradient steps: 0</div>
<div class=""> Gradient steps: 1</div>
<div class=""> nls ksp atol: 0</div>
<div class=""> nls ksp rtol: 1</div>
<div class=""> nls ksp ctol: 0</div>
<div class=""> nls ksp negc: 0</div>
<div class=""> nls ksp dtol: 0</div>
<div class=""> nls ksp iter: 0</div>
<div class=""> nls ksp othr: 0</div>
<div class=""> TaoLineSearch Object: 1 MPI processes</div>
<div class=""> type: armijo</div>
<div class=""> KSP Object: 1 MPI processes</div>
<div class=""> type: cg</div>
<div class=""> total KSP iterations: 21</div>
<div class=""> convergence tolerances: fatol=0, frtol=0</div>
<div class=""> convergence tolerances: gatol=0, steptol=0, gttol=0.0001</div>
<div class=""> Residual in Function/Gradient:=0.038741</div>
<div class=""> Objective value=0.639121</div>
<div class=""> total number of iterations=0, (max: 50)</div>
<div class=""> total number of function evaluations=31, max: 10000</div>
<div class=""> total number of gradient evaluations=1, max: 10000</div>
<div class=""> total number of function/gradient evaluations=1, (max: 10000)</div>
<div class=""> total number of Hessian evaluations=1</div>
<div class=""> Solver terminated: -6 Line Search Failure</div>
</div>
</div>
<div class=""><br class="">
</div>
<div class=""><b class="">This is without line-search (unit step size):</b></div>
<div class=""><br class="">
</div>
<div class="">
<div class="">Tao Object: 1 MPI processes</div>
<div class=""> type: nls</div>
<div class=""> Newton steps: 3</div>
<div class=""> BFGS steps: 0</div>
<div class=""> Scaled gradient steps: 0</div>
<div class=""> Gradient steps: 0</div>
<div class=""> nls ksp atol: 0</div>
<div class=""> nls ksp rtol: 3</div>
<div class=""> nls ksp ctol: 0</div>
<div class=""> nls ksp negc: 0</div>
<div class=""> nls ksp dtol: 0</div>
<div class=""> nls ksp iter: 0</div>
<div class=""> nls ksp othr: 0</div>
<div class=""> TaoLineSearch Object: 1 MPI processes</div>
<div class=""> type: unit</div>
<div class=""> KSP Object: 1 MPI processes</div>
<div class=""> type: cg</div>
<div class=""> total KSP iterations: 71</div>
<div class=""> convergence tolerances: fatol=0, frtol=0</div>
<div class=""> convergence tolerances: gatol=0, steptol=0, gttol=0.0001</div>
<div class=""> Residual in Function/Gradient:=1.91135e-11</div>
<div class=""> Objective value=0.160914</div>
<div class=""> total number of iterations=3, (max: 50)</div>
<div class=""> total number of function/gradient evaluations=4, (max: 10000)</div>
<div class=""> total number of Hessian evaluations=3</div>
<div class=""> Solution converged: ||g(X)||/||g(X0)|| <= gttol</div>
</div>
</div>
<div class=""><br class="">
</div>
<div class=""> </div>
<br class="">
</div>
</div>
<div class="">
<blockquote type="cite" class="">
<div class="">
<div class="">
<div class="">On Jun 11, 2015, at 12:44 PM, Jason Sarich <<a href="mailto:jason.sarich@gmail.com" target="_blank" class="">jason.sarich@gmail.com</a>> wrote:</div>
<br class="">
</div>
</div>
<div class="">
<div class="">
<div class="">
<div dir="ltr" class="">Hi Andreas,
<div class=""><br class="">
</div>
<div class="">Yes it looks like a bug that the reason is never set, but the line should still terminate. Is the problem you are having with the line search itself, or is it failing because you are checking this ls->reason directly?</div>
<div class=""><br class="">
</div>
<div class="">Jason Sarich</div>
<div class=""><br class="">
</div>
</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">On Thu, Jun 11, 2015 at 9:53 AM, Andreas Mang <span dir="ltr" class="">
<<a href="mailto:andreas@ices.utexas.edu" target="_blank" class="">andreas@ices.utexas.edu</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi guys:<br class="">
<br class="">
I have a problem with the TAO Armijo line search (petsc-3.5.4). My algorithm works if I use the More & Thuente line search (default). I have numerically checked the gradient of my objective. It’s correct. I am happy to write a small snippet of code and do an
easy test if you guys disagree, but from what I’ve seen in the line search code it seems obvious to me that there is a bug. Am I missing something or are you not setting<br class="">
<br class="">
ls->reason<br class="">
<br class="">
to<br class="">
<br class="">
TAOLINESEARCH_SUCCESS<br class="">
<br class="">
if the Armijo condition is fulfilled (TaoLineSearchApply_Armijo in armijo.c; line 118 - 302)?!<br class="">
<br class="">
It seems to me that ls->reason is and will remain to be set to<br class="">
<br class="">
TAOLINESEARCH_CONTINUE_ITERATING<br class="">
<br class="">
if everything works (i.e. I don’t hit one of the exceptions). Does this make sense? If not I’ll invest the time and put together a simple test case and, if that works, continue to check my code.<br class="">
<span class=""><font color="#888888" class=""><br class="">
/Andreas<br class="">
<br class="">
<br class="">
</font></span></blockquote>
</div>
<br class="">
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div></div></div>
</div>
</blockquote></div><br class=""></div>
</div></blockquote></div><br class=""></div></body></html>