Abstract
We propose a variant of the Nelder-Mead algorithm derived from a reinterpretation of univariate golden-section direct search. In the univariate case, convergence of the variant can be analyzed analogously to golden-section search. In the multivariate case, we modify the variant by replacing strict descent with fortified descent and maintaining the interior angles of the simplex bounded away from zero. Convergence of the modified variant can be analyzed by applying results for a fortified-descent simplicial search method. Some numerical experience with the variant is reported.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
D.P. Bertsekas, Nonlinear Programming, 2nd edn., Athena Scientific: Belmont, 1999.
J.E. Dennis, Jr. and V. Torczon, “Direct search methods on parallel machines,” SIAM Journal on Optimization, vol. 1, pp. 448–474, 1991.
R. Fletcher, Practical Methods of Optimization, vol. 1, John Wiley: New York, 1980.
C.T. Kelly, Iterative Methods for Optimization, SIAM: Philadelphia, PA, 1999.
A.G. Kuznetsov, “Nonlinear optimization toolbox,” Department of Engineering Science, University of Oxford, Oxford, UK, Report OUEL 1936/92, 1992.
J.C. Lagarias, J.A. Reeds, M.H. Wright, and P.E. Wright, “Convergence properties of the Nelder-Mead simplex algorithm in low dimensions,” SIAM Journal on Optimization, vol. 9, pp. 112–147, 1998.
D.G. Luenberger, Linear and Nonlinear Programming, 2nd edn., Addison-Wesley: Reading, MA, 1984.
O.L. Mangasarian, Nonlinear Programming, McGraw-Hill: New York, 1969.
K.I.M. McKinnon, “Convergence of the Nelder-Mead simplex method to a non-stationary point,” SIAM Journal on Optimization, vol. 9, pp. 148–158, 1998.
J.J. Moré, B.S. Garbow, and K.E. Hillstrom, “Testing unconstrained optimization software,” ACM Transactions on Mathematical Software, vol. 7, pp. 17–41, 1981.
J.A. Nelder and R. Mead, “A simplex method for function minimization,” Computer Journal, vol. 7, pp. 308–313, 1965.
M.J.D. Powell, “An efficient method for finding the minimum of a function of several variables without calculating derivatives,” Computer Journal, vol. 7, pp. 155–162, 1964.
M.J.D. Powell, “Direct search algorithms for optimization calculations,” Acta Numerica, vol. 7, pp. 287–336, 1998.
W. Spendley, G.R. Hext, and F.R. Himsworth, “Sequential application of simplex designs in optimisation and evolutionary operation,” Technometrica, vol. 4, pp. 441–461, 1962.
V. Torczon, “Multi-directional search: A direct search algorithm for parallel machines,” Ph.D. Thesis, Rice University, Houston, TX, 1989; available as Department of Mathematical Sciences, Rice University, Houston, TX, Technical Report 90-7.
P. Tseng, “Fortified-descent simplicial search method: A general approach,” SIAM Journal on Optimization, vol. 10, pp. 269–288, 2000.
M.H. Wright, “Direct search methods: Once scorned, now respectable,” in Numerical Analysis 1995, Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis, D.F. Griffiths and G.A. Watson (Eds.), Addison-Wesley Longman: Harlow, UK, 1996, pp. 191–208.
W.I. Zangwill, “Minimizing a function without calculating derivatives,” Computer Journal, vol. 10, pp. 293–296, 1967.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Nazareth, L., Tseng, P. Gilding the Lily: A Variant of the Nelder-Mead Algorithm Based on Golden-Section Search. Computational Optimization and Applications 22, 133–144 (2002). https://doi.org/10.1023/A:1014842520519
Issue Date:
DOI: https://doi.org/10.1023/A:1014842520519