Convergence of the Restricted Nelder--Mead Algorithm in Two Dimensions
Name
Lagarias-2012-Convergence of the Restricted Nelder--Mead Algorithm in Two Dimensions.pdf
Size
713.05 KB
Format
Adobe PDF
Checksum (MD5)
be8c33d8b96a94d63950496b77970f97
Author(s)
Lagarias, Jeffrey C.
Poonen, Bjorn
Wright, Margaret H.
Date Issued
May 2012
Journal
SIAM Journal on Optimization
Publisher
Society for Industrial and Applied Mathematics
Citation
Lagarias, Jeffrey C., Bjorn Poonen, and Margaret H. Wright. “Convergence of the Restricted Nelder--Mead Algorithm in Two Dimensions.” SIAM Journal on Optimization 22.2 (2012): 501–532. © 2012 SIAM
Version
Final published version
Abstract
The Nelder--Mead algorithm, a longstanding direct search method for unconstrained optimization published in 1965, is designed to minimize a scalar-valued function $f$ of $n$ real variables using only function values, without any derivative information. Each Nelder--Mead iteration is associated with a nondegenerate simplex defined by $n + 1$ vertices and their function values; a typical iteration produces a new simplex by replacing the worst vertex by a new point. Despite the method's widespread use, theoretical results have been limited: for strictly convex objective functions of one variable with bounded level sets, the algorithm always converges to the minimizer; for such functions of two variables, the diameter of the simplex converges to zero but examples constructed by McKinnon show that the algorithm may converge to a nonminimizing point. This paper considers the restricted Nelder--Mead algorithm, a variant that does not allow expansion steps. In two dimensions we show that for any nondegenerate starting simplex and any twice-continuously differentiable function with positive definite Hessian and bounded level sets, the algorithm always converges to the minimizer. The proof is based on treating the method as a discrete dynamical system and relies on several techniques that are nonstandard in convergence proofs for unconstrained optimization.
MIT Department
Massachusetts Institute of Technology. Department of Mathematics
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1137/110830150
