[petsc-users] TAO: armijo condition not fulfilled
Jason Sarich
jason.sarich at gmail.com
Thu Jun 11 16:13:30 CDT 2015
Hi Andreas,
I'm pretty sure this is a bug on my side, directly related to the ls->reason
Can you try fixing src/tao/linesearch/impls/armijo/armijo.c so it sets this
correctly?
before (line 275):
/* Successful termination, update memory */
armP->lastReference = ref;
/* Successful termination, update memory */
ls->reason = TAOLINESEARCH_SUCCESS;
armP->lastReference = ref;
On Thu, Jun 11, 2015 at 3:26 PM, Andreas Mang <andreas at ices.utexas.edu>
wrote:
> Hey Jason:
>
> Thanks for looking into this. In the meantime I have checked against
> your elliptic tao example using an Armijo linesearch. It works (i.e.
> converges) as you suggested in an earlier email even though it returns the
> “wrong" flag. For my problem it does even choke if I set the regularization
> parameter to 1E6 (essentially solving a quadratic problem).
>
> A final question before I continue the struggle by myself: It says
> “gradient steps: 1” instead of “gradient steps: 0” in the outputs (Armijo
> vs. More-Thuente / Unit). Does it start doing gradient evaluations? Maybe
> this helps me to further poke my code.
>
> I’ll continue to look into this. I’ll come back to you if I discover
> that the problem is on the PETSc side of things and I can reproduce the
> problem with a toy example.
>
> Thanks for your time! /Andreas
>
>
> On Jun 11, 2015, at 3:03 PM, Jason Sarich <jason.sarich at gmail.com> wrote:
>
> Hi Andreas,
>
> I don't see anything obviously wrong. If the function is very flat, you
> can try setting -tao_ls_armijo_sigma to a smaller number. If you continue
> to have problems, please let me know. It would definitely help if you have
> an example you could send me that reproduces this behavior.
>
> Jason
>
>
>
>
> On Thu, Jun 11, 2015 at 1:12 PM, Andreas Mang <andreas at ices.utexas.edu>
> wrote:
>
>> Hey Jason:
>>
>> The line search fails. If I use Armijo I get
>>
>> TaoLineSearch Object:
>> type: armijo
>> maxf=30, ftol=1e-10, gtol=0.0001
>> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001
>> memsize=1
>> maximum function evaluations=30
>> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9
>> total number of function evaluations=1
>> total number of gradient evaluations=1
>> total number of function/gradient evaluations=0
>> Termination reason: 0
>>
>> The parameters seem to be the default ones also suggested by Nocedal
>> and Wright. So I did not change anything. The termination reason is
>> equivalent to TAOLINESEARCH_CONTINUE_ITERATING. I am not checking the
>> reason directly. I guess it starts reducing the step size after that. I can
>> see that my objective function get’s evaluated (as expected); however, the
>> objective values increase (from what I see when monitoring the evaluations
>> of my objective). This leads to a failure in the line search and made
>> (still makes) me believe there is a bug on my side (which I have not found
>> yet). However, if I use a unit step it converges (relative change of the
>> gradient e.g. to 1E-9; see bottom of this email). If I use More & Thuente,
>> same thing. No reduction in step size necessary.
>>
>> If you suggest that I should do some further testing on simpler
>> problems, I’m happy to do so. After looking at the code, I just felt like
>> there obviously is something wrong in the line-search implementation.
>>
>> Thanks for your help.
>> /Andreas
>>
>> *Here’s the output after the first iteration (where the Armijo line
>> search fails):*
>>
>> TaoLineSearch Object:
>> type: armijo
>> maxf=30, ftol=1e-10, gtol=0.0001
>> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001
>> memsize=1
>> maximum function evaluations=30
>> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9
>> total number of function evaluations=1
>> total number of gradient evaluations=1
>> total number of function/gradient evaluations=0
>> Termination reason: 0
>> TaoLineSearch Object:
>> type: armijo
>> maxf=30, ftol=1e-10, gtol=0.0001
>> Armijo linesearch : alpha=1 beta=0.5 sigma=0.0001
>> memsize=1
>> maximum function evaluations=30
>> tolerances: ftol=0.0001, rtol=1e-10, gtol=0.9
>> total number of function evaluations=30
>> total number of gradient evaluations=0
>> total number of function/gradient evaluations=0
>> Termination reason: 4
>>
>> *With final output (end of optimization):*
>>
>> Tao Object: 1 MPI processes
>> type: nls
>> Newton steps: 1
>> BFGS steps: 0
>> Scaled gradient steps: 0
>> Gradient steps: 1
>> nls ksp atol: 0
>> nls ksp rtol: 1
>> nls ksp ctol: 0
>> nls ksp negc: 0
>> nls ksp dtol: 0
>> nls ksp iter: 0
>> nls ksp othr: 0
>> TaoLineSearch Object: 1 MPI processes
>> type: armijo
>> KSP Object: 1 MPI processes
>> type: cg
>> total KSP iterations: 21
>> convergence tolerances: fatol=0, frtol=0
>> convergence tolerances: gatol=0, steptol=0, gttol=0.0001
>> Residual in Function/Gradient:=0.038741
>> Objective value=0.639121
>> total number of iterations=0, (max: 50)
>> total number of function evaluations=31, max: 10000
>> total number of gradient evaluations=1, max: 10000
>> total number of function/gradient evaluations=1, (max: 10000)
>> total number of Hessian evaluations=1
>> Solver terminated: -6 Line Search Failure
>>
>> *This is without line-search (unit step size):*
>>
>> Tao Object: 1 MPI processes
>> type: nls
>> Newton steps: 3
>> BFGS steps: 0
>> Scaled gradient steps: 0
>> Gradient steps: 0
>> nls ksp atol: 0
>> nls ksp rtol: 3
>> nls ksp ctol: 0
>> nls ksp negc: 0
>> nls ksp dtol: 0
>> nls ksp iter: 0
>> nls ksp othr: 0
>> TaoLineSearch Object: 1 MPI processes
>> type: unit
>> KSP Object: 1 MPI processes
>> type: cg
>> total KSP iterations: 71
>> convergence tolerances: fatol=0, frtol=0
>> convergence tolerances: gatol=0, steptol=0, gttol=0.0001
>> Residual in Function/Gradient:=1.91135e-11
>> Objective value=0.160914
>> total number of iterations=3, (max: 50)
>> total number of function/gradient evaluations=4, (max: 10000)
>> total number of Hessian evaluations=3
>> Solution converged: ||g(X)||/||g(X0)|| <= gttol
>>
>>
>>
>> On Jun 11, 2015, at 12:44 PM, Jason Sarich <jason.sarich at gmail.com>
>> wrote:
>>
>> Hi Andreas,
>>
>> Yes it looks like a bug that the reason is never set, but the line
>> should still terminate. Is the problem you are having with the line search
>> itself, or is it failing because you are checking this ls->reason directly?
>>
>> Jason Sarich
>>
>>
>> On Thu, Jun 11, 2015 at 9:53 AM, Andreas Mang <andreas at ices.utexas.edu>
>> wrote:
>>
>>> Hi guys:
>>>
>>> I have a problem with the TAO Armijo line search (petsc-3.5.4). My
>>> algorithm works if I use the More & Thuente line search (default). I have
>>> numerically checked the gradient of my objective. It’s correct. I am happy
>>> to write a small snippet of code and do an easy test if you guys disagree,
>>> but from what I’ve seen in the line search code it seems obvious to me that
>>> there is a bug. Am I missing something or are you not setting
>>>
>>> ls->reason
>>>
>>> to
>>>
>>> TAOLINESEARCH_SUCCESS
>>>
>>> if the Armijo condition is fulfilled (TaoLineSearchApply_Armijo in
>>> armijo.c; line 118 - 302)?!
>>>
>>> It seems to me that ls->reason is and will remain to be set to
>>>
>>> TAOLINESEARCH_CONTINUE_ITERATING
>>>
>>> if everything works (i.e. I don’t hit one of the exceptions). Does this
>>> make sense? If not I’ll invest the time and put together a simple test case
>>> and, if that works, continue to check my code.
>>>
>>> /Andreas
>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150611/1906646f/attachment.html>
More information about the petsc-users
mailing list