Abstract
The pertinent literature controversially discusses in which respects evolutionary algorithms differ from classical gradient methods. This chapter presents a hybrid, called the evolutionary-gradient-search procedure, that uses evolutionary variations to estimate the gradient direction in which it then performs an optimization step. Both standard benchmarks and theoretical analyses suggest that this hybrid yields superior performance. In addition, this chapter presents inverse mutation, a new concept that proves particularly useful in the presence of noise, which is omnipresent in almost any real-world application.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arnold, D.: An analysis of evolutionary gradient search. In: Proceedings of the Congress on Evolutionary Computation (CEC 2004), pp. 47–54. IEEE Press, Los Alamitos (2004)
Arnold, D., Salomon, R.: Evolutionary gradient search revisited. Techreport CS-2005-09, Faculty of Computer Science, Dalhousie University (2005)
Arnold, D., Beyer, H.G.: Local performance of the (μ/μ,λ)-Evolution Strategy in a noisy environment. In: Martin, W.N., Spears, W.M. (eds.) Proceeding of Foundation of Genetic Algorithms 6 (FOGA 2006), pp. 127–141. Morgan Kaufmann, San Francisco (2001)
Arnold, D., Salomon, R.: Evolutionary gradient search revisited. IEEE Transactions on Evolutionary Computation 11(4), 480–495 (2007)
Bäck, T., Schwefel, H.P.: An overview of evolutionary algorithms for parameter optimization. Evolutionary Computation 1(1), 1–23 (1993)
Beyer, H.G.: An alternative explanation for the manner in which genetic algorithms operate. BioSystems 41, 1–15 (1997)
Beyer, H.G.: The Theory of Evolution Strategies. Springer, Heidelberg (2001)
Fogel, D.B.: Evolutionary Computation: Toward a New Philosophy of Machine Learning Intelligence. IEEE Press, Piscataway (1995)
Fogel, L.J.: Autonomous automata. Industrial Research 4, 14–19 (1962)
Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading (1989)
Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)
Luenberger, D.G.: Linear and Nonlinear Programming. Addison-Wesley, Reading (1984)
Ostermeier, A., Gawelczyk, A., Hansen, N.: Step-size adaptation based on non-local use of selection information. In: Davidor, Y., änner, R.M., Schwefel, H.P. (eds.) Proceedings of the 3rd International Conference on Parallel Problem Solving from Nature (PPSN III), pp. 189–198. Springer, Heidelberg (1994)
Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C. Cambridge University Press, Cambridge (1994)
Rechenberg, I.: Evolutionsstrategie 1994. Frommann-Holzboog, Stuttgart (1994)
Rudolph, G.: On correlated mutations in evolution strategies. In: änner, R.M., Manderick, B. (eds.) Proceedings of the 2nd International Conference on Parallel Problem Solving from Nature (PPSN II), pp. 105–114. Elsevier, Amsterdam (1992)
Rumelhart, et al. (eds.): Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 2. MIT Press, Cambridge (1986)
Salomon, R.: Evolutionary algorithms and gradient search: similarities and differences. IEEE Transactions on Evolutionary Computation 2(2), 45–55 (1998)
Salomon, R.: Accelerating the Evolutionary-Gradient-Search procedure: Individual step sizes. In: Eiben, A.E., Bäck, T., Schoenauer, M., Schwefel, H.P. (eds.) Proceedings of the 5th International Conference on Parallel Problem Solving from Nature (PPSN V), pp. 408–417. Springer, Heidelberg (1998)
Salomon, R., van Hemmen, J.L.: Accelerating backpropagation through dynamic self-adaptation. Neural Networks 9(4), 589–601 (1996)
Schwefel, H.P.: Evolution and Optimum Seeking. John Wiley and Sons, Chichester (1995)
Schwefel, H.P.: Evolutionary Computation — A Study on Collective Learning. In: Callaos, N., Khoong, C.M., Cohen, E. (eds.) Proceedings of the World Multiconference on Systemics, Cybernetics and Informatics, Orlando, FL, USA. International Institute of Informatics and Systemics, vol. 2, pp. 198–205 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Salomon, R., Arnold, D.V. (2009). The Evolutionary-Gradient-Search Procedure in Theory and Practice. In: Chiong, R. (eds) Nature-Inspired Algorithms for Optimisation. Studies in Computational Intelligence, vol 193. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00267-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-00267-0_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-00266-3
Online ISBN: 978-3-642-00267-0
eBook Packages: EngineeringEngineering (R0)