<div class="gmail_quote">On Tue, May 15, 2012 at 2:41 PM, behzad baghapour <span dir="ltr"><<a href="mailto:behzad.baghapour@gmail.com" target="_blank">behzad.baghapour@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks. Of course, the Line-search just change the step length. But I could not get What you mean that the residual must be increased before reaching the steady-state?</blockquote></div><br><div>This is the key to pseudotransient continuation's global convergence. If steady state could be reached by simply minimizing the residual, then steepest descent (and trust region Newton) would be globally convergent. It would also render optimization research obsolete because nonlinear equation solvers could naively be applied to the first order optimality conditions without concern for the objective. Alas, it doesn't work that way: steady state often cannot be reached without increasing the steady-state residual and non-convex optimization problems require that the gradient be allowed to increase.</div>
<div><br></div><div>This is perhaps easier to illustrate for the optimization case. Consider</div><div><br></div><div>f(x) = x^2 - exp(-4*(x-2)^2)</div><div><br></div><div>which is a nice-looking function with a single minimum very close to x=0.</div>
<div><br></div><div><a href="http://www.wolframalpha.com/input/?i=x%5E2+-+exp%28-4*%28x-2%29%5E2%29">http://www.wolframalpha.com/input/?i=x%5E2+-+exp%28-4*%28x-2%29%5E2%29</a></div><div><br></div><div>Now the gradient clearly has these local minima, so searching in the norm of the gradient will get stuck at a point that is not a local minima.</div>
<div><br></div><div><a href="http://www.wolframalpha.com/input/?i=diff%28x%5E2+-+exp%28-4*%28x-2%29%5E2%29%2Cx%29">http://www.wolframalpha.com/input/?i=diff%28x%5E2+-+exp%28-4*%28x-2%29%5E2%29%2Cx%29</a></div><div><br></div>
<div><br></div><div>For transient problems, there are lots of reasons that the residual may need to grow before steady state can be reached. See, for example, Figure 3.1 and 3.2 of <a href="http://www.cs.odu.edu/~keyes/papers/ptc03.pdf">http://www.cs.odu.edu/~keyes/papers/ptc03.pdf</a> . Note that the residual is not monotone. Although they do not prove it here, steady state for these problems cannot be reached without hill climbing: inherently transient processes like combustion must take place before a steady state can be reached.</div>