Abstract
Probably the most popular algorithm for unconstrained minimization for problems of moderate dimension is the Nelder-Mead algorithm published in 1965. Despite its age only limited convergence results exist. Several counterexamples can be found in the literature for which the algorithm performs badly or even fails. A convergent variant derived from the original Nelder-Mead algorithm is presented. The proposed algorithm's convergence is based on the principle of grid restrainment and therefore does not require sufficient descent as the recent convergent variant proposed by Price, Coope, and Byatt. Convergence properties of the proposed grid-restrained algorithm are analysed. Results of numerical testing are also included and compared to the results of the algorithm proposed by Price et al. The results clearly demonstrate that the proposed grid-restrained algorithm is an efficient direct search method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
D. Byatt, “A convergent variant of the Nelder-Mead algorithm,” Master’;s thesis, Mathematics and Statistics Department, University of Canterbury, Christchurch, NZ, 2000.
I.D. Coope and C.J. Price, “Frame based methods for unconstrained optimization,” Journal of Optimization Theory and Applications, vol. 107, pp. 261–274, 2000.
I.D. Coope and C.J. Price, “On the convergence of grid-based methods for unconstrained optimization,” SIAM J. Optim., vol. 11, pp. 859–869, 2001.
C. Davis, “Theory of positive linear dependence,” American Journal of Mathematics, vol. 76, pp. 733–746, 1954.
C.T. Kelley, “Detection and remediation of stagnation in the Nelder-Mead algorithm using a sufficient decrease condition,” SIAM Journal on Optimization, vol. 10, pp. 43–55, 1999.
J.C. Lagarias, “Convergence properties of the Nelder-Mead simplex method in low dimensions,” SIAM Journal on Optimization, vol. 9, pp. 112–147, 1998.
R.M. Lewis and V.J. Torczon, “Rank ordering and positive bases in pattern search algorithms,” ICASE NASA Langley Research Center, Hampton, VA 23681-0001, USA, Tech. Report 96-71, 1996.
R.M. Lewis, V.J. Torczon, and M.W. Trosset, “Direct search methods: Then and now,” Journal of Computational and Applied Mathematics, vol. 124, pp. 191–207, 2000.
K.I.M. McKinnon, “Convergence of the Nelder-Mead simplex method,” SIAM Journal on Optimization, vol. 9, pp. 148–158, 1998.
J.J. Moré, B.S. Garbow, and K.E. Hillstrom, “Testing unconstrained optimization software,” ACM Transactions on Mathematical Software, vol. 7, pp. 17–41, 1981.
J.A. Nelder and R. Mead, “A simplex method for function minimization,” The Computer Journal, vol. 7, pp. 308–313, 1965.
C.J. Price, I.D. Coope, and D. Byatt, “A convergent variant of the Nelder-Mead algorithm,” Journal of Optimization Theory and Applications, vol. 113, pp. 5–19, 2002.
A.S. Rykov, “Simplex algorithms for unconstrained minimization,” Problems of Control and Information Theory, vol. 12, pp. 195–208, 1983.
V.J. Torczon, “Multi-directional search: A direct search algorithm for parallel machines,” PhD thesis, Department of Mathematical Sciences, Rice University, Houston, TX 77251-1892, USA, 1989.
V.J. Torczon, “On the convergence of pattern search methods,” SIAM J. Optim., vol. 7, pp. 1–25, 1997.
P. Tseng, “Fortified-descent simplical search method: A general approach,” SIAM Journal on Optimization, vol. 10, pp. 269–288, 1999.
M.H. Wright, “Direct search methods: Once scorned, now respectable,” in Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis, Dundee, UK, 1996, pp. 191–208.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bűrmen, Á., Puhan, J. & Tuma, T. Grid Restrained Nelder-Mead Algorithm. Comput Optim Applic 34, 359–375 (2006). https://doi.org/10.1007/s10589-005-3912-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-005-3912-z