[petsc-dev] TAO convergence tests are undecipherable

[Jason Sarich]

>
> The function tolerances are usually something like |f(x_k) - f_(x_{k-1})|
> <= fatol


But that is *not* what they are in TAO, this has always been a source of
confusion.

The one upside of the f-tolerances is that they give the users a reasonable
level of confidence that the objective function value
achieved is within ftol of the true minimum, but it just happens to be
redundant because f(x)-f(X*) is estimated using ||g(x)||^2 (How? I thought
this made sense to me once, but now I don't see it anymore) and we also
have a convergence criteria for ||g(x)||.



On Wed, Sep 9, 2015 at 10:02 AM, Munson, Todd S. <tmunson at mcs.anl.gov>
wrote:

>
> I do not see a need to keep them around.  The function tolerances are
> usually something like |f(x_k) - f_(x_{k-1})| <= fatol and then you
> use |f(x_0)| in the denominator for the relative tolerance.  There
> are pretty useless tests though and give many false positives.
>
> Todd.
>
> > On Sep 9, 2015, at 9:26 AM, Jason Sarich <sarich at mcs.anl.gov> wrote:
> >
> > Todd, can you come up with a reason why we might need to keep these
> fatol, frtol tests around? I would prefer to take them out.
> >
> > Jason
> >
> > On Tue, Sep 8, 2015 at 9:54 PM, Smith, Barry F. <bsmith at mcs.anl.gov>
> wrote:
> >
> >   Jason,
> >
> >    Perhaps it is just documentation.
> >
> > /*@
> >   TaoSetTolerances - Sets parameters used in TAO convergence tests
> >
> >   Logically collective on Tao
> >
> >   Input Parameters:
> > + tao - the Tao context
> > . fatol - absolute convergence tolerance
> > . frtol - relative convergence tolerance
> > . gatol - stop if norm of gradient is less than this
> > . grtol - stop if relative norm of gradient is less than this
> > - gttol - stop if norm of gradient is reduced by this factor
> >
> >   Options Database Keys:
> > + -tao_fatol <fatol> - Sets fatol
> > . -tao_frtol <frtol> - Sets frtol
> > . -tao_gatol <gatol> - Sets gatol
> > . -tao_grtol <grtol> - Sets grtol
> > - -tao_gttol <gttol> - Sets gttol
> >
> >   Stopping Criteria:
> > $ f(X) - f(X*) (estimated)            <= fatol
> > $ |f(X) - f(X*)| (estimated) / |f(X)| <= frtol
> > $ ||g(X)||                            <= gatol
> > $ ||g(X)|| / |f(X)|                   <= grtol
> > $ ||g(X)|| / ||g(X0)||                <= gttol
> >
> > How is f(x) - f(X*) (estimated) and how come the second definition has
> absolute values but the first does not?
> >
> > From below it looks like f(x) - f(X*)  is estimated as gnorm*gnorm?
> >
> >   } else if (gnorm2 <= fatol && cnorm <=catol) {
> >     ierr = PetscInfo2(tao,"Converged due to estimated f(X) - f(X*) = %g
> < %g\n",(double)gnorm2,(double)fatol);CHKERRQ(ierr);
> >     reason = TAO_CONVERGED_FATOL;
> >   } else if (f != 0 && gnorm2 / PetscAbsReal(f)<= frtol &&
> cnorm/PetscMax(cnorm0,1.0) <= crtol) {
> >     ierr = PetscInfo2(tao,"Converged due to estimated |f(X)-f(X*)|/f(X)
> = %g < %g\n",(double)(gnorm2/PetscAbsReal(f)),(double)frtol);CHKERRQ(ierr);
> >     reason = TAO_CONVERGED_FRTOL;
> >
> > It seems the next test is the same as the first I list above exact gnorm
> is not squared? Why is it worth having this duplicate test?
> >
> >   } else if (gnorm<= gatol && cnorm <=catol) {
> >     ierr = PetscInfo2(tao,"Converged due to residual norm ||g(X)||=%g <
> %g\n",(double)gnorm,(double)gatol);CHKERRQ(ierr);
> >     reason = TAO_CONVERGED_GATOL;
> >
> > At a minimum the manual page should document what f(X) - f(X*)
> (estimated)     means.
> >
> >   Barry
> >
> >   } else if ( f!=0 && PetscAbsReal(gnorm/f) <= grtol && cnorm <= crtol) {
> >     ierr = PetscInfo2(tao,"Converged due to residual ||g(X)||/|f(X)| =%g
> < %g\n",(double)(gnorm/f),(double)grtol);CHKERRQ(ierr);
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20150909/2c2d7bf2/attachment.html>