Abstract
It is a well known fact that the widely used Nelder–Mead (NM) optimization method soon becomes inefficient as the dimension of the problem grows. It has been observed that part of the reason for the inefficiency is that the search direction becomes practically perpendicular to the direction of the local downhill gradient. In this paper, we identify which operations are responsible for the increase of the angle between the search direction and the direction of the local downhill gradient. We show that it is possible to decrease this angle by randomly perturbing the centroid of the n best vertices around which the original NM algorithm moves the worst vertex. Using a perturbed centroid, the algorithm outperforms the standard NM method as well as the NM method with adaptive parameters for problems with higher dimensions, and works well for problems with dimensions of well over 100.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Blum, A., Hopcroft, J., Kannan, R.: Foundations of Data Science. Unpublished (2017) https://www.cs.cornell.edu/jeh/book.pdf
Duarte, A., Martí, R., Gortazar, F.: Path relinking for large-scale global optimization. Soft Comput. 15(11), 2257–2273 (2011). https://doi.org/10.1007/s00500-010-0650-7
Fajfar, I., Puhan, J., Bürmen, Á.: Evolving a nelder-mead algorithm for optimization with genetic programming. Evolut. Comput. 25(3), 351–373 (2017). https://doi.org/10.1162/evco_a_00174
Gao, F., Han, L.: Implementing the Nelder–Mead simplex algorithm with adaptive parameters. Comput. Optim. Appl. 51(1), 259–277 (2012). https://doi.org/10.1007/s10589-010-9329-3
Han, L., Neumann, M.: Effect of dimensionality on the Nelder–Mead simplex method. Optim. Methods Softw. 21(1), 1–16 (2006). https://doi.org/10.1080/10556780512331318290
Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.E.: Convergence properties of the Nelder–Mead simplex method in low dimensions. SIAM J. Optim. 9(1), 112–147 (1998)
McKinnon, K.I.M.: Convergence of the Nelder–Mead simplex method to a nonstationary point. SIAM J. Optim. 9(1), 148–158 (1998)
Mohapatra, P., Nath Das, K., Roy, S.: A modified competitive swarm optimizer for large scale optimization problems. Appl. Soft Comput. 59(C), 340–362 (2017). https://doi.org/10.1016/j.asoc.2017.05.060
Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 17–41 (1981)
Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965)
Torczon, V.J.: Multi-directional search: A direct search algorithm for parallel machines. Ph.D. thesis, Rice University, Houston, TX (1989)
Wright, M.H.: Direct search methods: once scorned, now respectable. In: Griffiths DF, Watson GA (eds) Numerical Analysis 1995, In: Proceedings of the 1995 dundee biennial conference in numerical analysis, CRC Press, Boca Raton, FL, Pitman Research Notes in Mathematics, vol 344, pp. 191–208 (1996). http://www.crcpress.com/shopping_cart/products/product_detail.asp?sku=LM7633&parent_id=&pc=/shopping_cart/search/search.asp!
Yang, P., Tang, K., Yao, X.: Turning high-dimensional optimization into computationally expensive optimization. IEEE Trans. Evolut. Comput. 22(1), 143–156 (2018). https://doi.org/10.1109/TEVC.2017.2672689
Acknowledgements
This work was supported by the Ministry of Education, Science and Sport of Republic of Slovenia under Research program P2-0246—Algorithms and optimization methods in telecommunications.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fajfar, I., Bűrmen, Á. & Puhan, J. The Nelder–Mead simplex algorithm with perturbed centroid for high-dimensional function optimization. Optim Lett 13, 1011–1025 (2019). https://doi.org/10.1007/s11590-018-1306-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-018-1306-2