Overdetermined, non-linear

[Erlend Pedersen :.]

On Tue, 2008-02-05 at 07:31 -0600, Matthew Knepley wrote:
> On Feb 5, 2008 3:26 AM, Erlend Pedersen :.
> <erlend.pedersen at holberger.com> wrote:
> > On Sun, 2008-02-03 at 19:59 -0600, Matthew Knepley wrote:
> > > On Feb 1, 2008 5:54 AM, Erlend Pedersen :.
> > > <erlend.pedersen at holberger.com> wrote:
> > > > I am attempting to use the PETSc nonlinear solver on an overdetermined
> > > > system of non-linear equations. Hence, the Jacobian is not square, and
> > > > so far we have unfortunately not succeeded with any combination of snes,
> > > > ksp and pc.
> > > >
> > > > Could you confirm that snes actually works for overdetermined systems,
> > > > and if so, is there an application example we could look at in order to
> > > > make sure there is nothing wrong with our test-setup?
> > > >
> > > > We have previously used the MINPACK routine LMDER very successfully, but
> > > > for our current problem sizes we rely on the use of sparse matrix
> > > > representations and parallel architectures. PETSc's abstractions and
> > > > automatic MPI makes this system very attractive for us, and we have
> > > > already used the PETSc LSQR solver with great success.
> > >
> > > So in the sense that SNES is really just an iteration with an embedded solve,
> > > yes it can solve non-square nonlinear systems. However, the user has to
> > > understand what is meant by the Function and Jacobian evaluation methods.
> > > I suggest implementing the simplest algorithm for non-square systems:
> > >
> > > http://en.wikipedia.org/wiki/Gauss-Newton_algorithm
> > >
> > > By implement, I mean your Function and Jacobian methods should return the
> > > correct terms. I believe the reason you have not seen convergence is that
> > > the result of the solve does not "mean" the correct thing for the iteration
> > > in your current setup.
> > >
> > >    Matt
> >
> > Thanks. Good to know that I should be able to get a working setup. Are
> > there by any chance any code examples that I could use to clue myself in
> > on how to transform my m equations of n unknonwns into a correct
> > function for the Gauss-Newton algorithm?
> 
> We do not have any nonlinear least-squares examples, unfortunately. At that
> point, most users have gone over to formulating their problem directly as
> an optimization problem (which allows more flexibility than least squares) and
> have moved to TAO (http://www-unix.mcs.anl.gov/tao/) which does have
> examples, I believe, for optimization of this kind.
> 
> If you know that you only ever want to do least squares, and you want to solve
> the biggest, parallel problems, than stick with PETSc and build a nice
> Gauss-Newton
> (or Levenberg-Marquadt) solver. However, if you really want to solve a more
> general optimization problem, I recommend reformulating it now and moving
> to TAO. It is at least worth reading up on it.

Reformulating as an optimization problem does seem like the easier route
for now. I kept away from TAO in order to Keep It Simple, but now I see
that the opposite might be the case. I should be able to provide it with
a gradient, if not necessarily a Hessian. Thanks again :)

- Erlend :.

> 
>   Thanks,
> 
>      Matt
> 
> > - Erlend :.