Skip to main content
SpringerLink
Log in
Book cover

Handbook of Optimization pp 1–28Cite as

Dynamic Optimization Using Analytic and Evolutionary Approaches: A Comparative Review

  • Chapter

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 38))

Abstract

Solving a dynamic optimization problem means that the obtained results depend explicitly on time as a parameter. There are two major branches in which dynamic optimization occurs: (i) in dynamic programming and optimal control, and (ii) in dynamic fitness landscapes and evolutionary computation. In both fields, solving such problems is established practice while at the same time special and advanced aspects are still subject of research. In this chapter, we intend to give a comparative study of the two branches of dynamic optimization. We review both problem settings, define them, and discuss approaches for and issues in solving them. The main focus here is to highlight the connections and parallels. In particular, we show that optimal control problems can be understood as dynamic fitness landscapes, where for linear systems this relationship can even be expressed analytically.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ahmed–Ali, T., Mazenc, F., Lamnabhi–Lagarrigue, F.: Disturbance attenuation for discrete-time feedforward nonlinear systems. In: Aeyels, D., Lamnabhi–Lagarrigue, F., van der Schaft, A. (eds.) Stability and Stabilization of Nonlinear Systems, pp. 1–17. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  2. Al–Tamimi, A., Lewis, F.L., Abu-Khalaf, M.: Discrete–time nonlinear HJB solution using approximate dynamic programming: Convergence proof. IEEE Trans Syst., Man, & Cybern. Part B: Cybern. 38, 943–949 (2008)

    Article  Google Scholar 

  3. Al–Tamimi, A., Abu-Khalaf, M., Lewis, F.L.: Heuristic dynamic programming nonlinear optimal controller. In: Mellouk, A., Chebira, A. (eds.) Machine Learning, pp. 361–380. InTech, Rijeka (2009)

    Google Scholar 

  4. Arnold, D.V., Beyer, H.G.: Optimum tracking with evolution strategies. Evol. Comput. 14, 291–308 (2006)

    Article  Google Scholar 

  5. Artstein, Z.: Stabilization with relaxed controls. Nonlinear Anal. 7, 1163–1173 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bäck, T.: Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. Oxford University Press, New York (1996)

    MATH  Google Scholar 

  7. Beard, R.W., Saridis, G.N.: Approximate solutions to the time–invariant Hamilton–Jacobi–Bellman equation. J. Optim. Theory Appl. 96, 589–626 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bellman, R.E.: Dynamic Programming, p. 1957. Princeton University Press, Princeton (2010)

    MATH  Google Scholar 

  9. Bendtsen, C.N., Krink, T.: Dynamic memory model for non–stationary optimization. In: Fogel, D.B., El–Sharkawi, M.A., Yao, X., Greenwood, G., Iba, H., Marrow, P.I., Shackleton, M. (eds.) Proc. Congress on Evolutionary Computation, IEEE CEC 2002, pp. 145–150. IEEE Press, Piscataway (2002)

    Google Scholar 

  10. Benton, M.J.: When Life Nearly Died–The Greatest Mass Extinction of All Time. Thames & Hudson, London (2003)

    Google Scholar 

  11. Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. 1. Athena Scientific, Belmont (2005)

    MATH  Google Scholar 

  12. Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. 2. Athena Scientific, Belmont (2007)

    Google Scholar 

  13. Betts, J.T.: Practical Methods for Optimal Control using Nonlinear Programming. SIAM, Philadelphia (2001)

    MATH  Google Scholar 

  14. Bobbin, J., Yao, X.: Solving optimal control problems with a cost on changing control by evolutionary algorithms. In: Bäck, T., Michalewicz, Z., Yao, X. (eds.) Proc. 1997 IEEE International Conference on Evolutionary Computation (ICEC 1997), pp. 331–336. IEEE Press, Piscataway (1997)

    Chapter  Google Scholar 

  15. Borenstein, E., Meilijson, I., Ruppin, E.: The effect of phenotypic plasticity on evolution in multipeaked fitness landscapes. Jour. Evolut. Biology 19, 1555–1570 (2006)

    Article  Google Scholar 

  16. Bosman, P.A.N.: Learning and anticipation in online dynamic optimization. In: Yang, S., Ong, Y.S., Jin, Y. (eds.) Evolutionary Computation in Dynamic and Uncertain Environments, pp. 129–152. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  17. Boumaza, A.M.: Learning environment dynamics from self-adaptation. In: Yang, S., Branke, J. (eds.) GECCO Workshops 2005, pp. 48–54 (2005)

    Google Scholar 

  18. Branke, J.: Memory enhanced evolutionary algorithms for changing optimization problems. In: Angeline, P.J., Michalewicz, Z., Schoenauer, M., Yao, X., Zalzala, A. (eds.) Proc. Congress on Evolutionary Computation, IEEE CEC 1999, pp. 1875–1882. IEEE Press, Piscataway (1999)

    Google Scholar 

  19. Branke, J.: Evolutionary Optimization in Dynamic Environments. Kluwer Academic Publishers, Dordrecht (2001)

    Google Scholar 

  20. Branke, J., Kaußler, T., Schmidt, C., Schmeck, H.: A multi–population approach to dynamic optimization problems. In: Parmee, I.C. (ed.) Proc. of the 4th Int. Conf. on Adaptive Computing in Design and Manufacturing, pp. 299–308 (2000)

    Google Scholar 

  21. Bui, L.T., Branke, J., Abbass, H.A.: Diversity as a selection pressure in dynamic environments. In: Beyer, H.G., O’Reilly, U.M. (eds.) Proc. Genetic and Evolutionary Computation Conference (GECCO 2005), pp. 1557–1558. ACM Press, Seattle (2005)

    Chapter  Google Scholar 

  22. Chen, Z., Jagannathan, S.: Generalized Hamilton–Jacobi–Bellman formulation based neural network control of affine nonlinear discrete-time systems. IEEE Trans. Neural Networks 19, 90–106 (2008)

    Article  Google Scholar 

  23. Cobb, H.G.: An investigation into the use of hypermutation as an adaptive operator in genetic algorithms having continuouis, time–dependent nonstationary environments. Technical Report AIC-90-001, Naval Research Laboratory, Washington, USA (1990), http://handle.dtic.mil/100.2/ADA229159

  24. Defaweux, A., Lenaerts, T., van Hemert, J., Parent, J.: Complexity transitions in evolutionary algorithms: evaluating the impact of the initial population. In: Corne, D. (ed.) Proc. Congress on Evolutionary Computation, IEEE CEC 2005, pp. 2174–2181. IEEE Press, Piscataway (2005)

    Chapter  Google Scholar 

  25. The Darwin Correspondence Project, http://www.darwinproject.ac.uk/six-things-darwin-never-said (retrieved July 08, 2011)

  26. den Boer, P.J.: Natural selection or the non–survival of the non–fit. Acta Biotheoretica 47, 83–97 (1999)

    Article  Google Scholar 

  27. Drezewski, R., Siwik, L.: Agent–based multi–objective evolutionary algorithm with sexual selection. In: Wang, J., Liu, D., Feng, G., Michalewicz, Z. (eds.) Proc. 2008 IEEE Congress on Evolutionary Computation, pp. 3679–3684. IEEE Press, Piscataway (2008)

    Google Scholar 

  28. Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  29. Fogel, D.B.: Applying evolutionary programming to selected control problems. Computers & Mathematics with Applications 27, 89–104 (1994)

    Article  MATH  Google Scholar 

  30. Franks, S.J., Sim, S., Weis, A.E.: Rapid evolution of flowering time by an annual plant in response to a climate fluctuation. Proc. Natl. Acad. Sci. USA (PNAS) 104, 1278–1282 (2007)

    Article  Google Scholar 

  31. Freeman, R.A., Kokotovic, P.V.: Inverse optimality in robust stabilization. SIAM J. Control Optim. 34, 1365–1391 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  32. Futuyma, D.J.: Evolution. Sinauer Associates, Sunderland (2005)

    Google Scholar 

  33. Grefenstette, J.J.: Genetic algorithms for changing environments. In: Männer, R., Manderick, B. (eds.) Parallel Problem Solving from Nature–PPSN II, pp. 137–144. North Holland, Amsterdam (1992)

    Google Scholar 

  34. Haddad, W.M., Chellaboina, V.: Nonlinear Dynamical Systems and Control: A Lyapunov-Based Approach. Princeton University Press, Princeton (2008)

    MATH  Google Scholar 

  35. Haddad, W.M., Chellaboina, V.: Discrete–time nonlinear analysis and feedback control with nonquadratic performance criteria. J. Franklin Inst. 333B, 849–860 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  36. Haddad, W.M., Chellaboina, V., Fausz, J.L., Abdallah, C.T.: Optimal discrete–time control for nonlinear cascade systems. J. Franklin Inst. 335B, 827–839 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  37. Hoffmann, A.A., Will, Y.: Detecting genetic responses to environmental change. Nat. Rev. Genet. 9, 421–432 (2008)

    Article  Google Scholar 

  38. Hu, X., Eberhart, R.C.: Adaptive particle swarm optimization: detection and response to dynamic systems. In: Fogel, D.B., El–Sharkawi, M.A., Yao, X., Greenwood, G., Iba, H., Marrow, P.I., Shackleton, M. (eds.) Proc. 2002 IEEE Congress on Evolutionary Computation, pp. 1666–1670. IEEE Press, Piscataway (2002)

    Google Scholar 

  39. Jablonka, E., Oborny, B., Molnar, E., Kisdi, E., Hofbauer, J., Czaran, T.: The adaptive advantage of phenotypic memory. Philosophical Transactions of the Royal Society, London B350, 133–141 (1995)

    Article  Google Scholar 

  40. Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments – A survey. IEEE Trans. Evolut. Comput. 9, 303–317 (2005)

    Article  Google Scholar 

  41. Kashtan, N., Noor, E., Alon, U.: Varying environments can speed up evolution. Proc. Natl. Acad. Sci. USA (PNAS) 104, 13711–13716 (2007)

    Article  Google Scholar 

  42. Kirschner, M.W., Gerhart, J.C.: The Plausibility of Life: Resolving Darwin’s Dilemma. Yale Univ. Press, New Haven (2005)

    Google Scholar 

  43. Levins, R.: Evolution in Changing Environments. Princeton University Press, Princeton (1968)

    Google Scholar 

  44. Li, X., Yong, J.: Optimal Control Theory for Infinite Dimensional Systems. Birkhäuser, Boston (1995)

    Book  Google Scholar 

  45. Lewis, J., Hart, E., Ritchie, G.: A Comparison of Dominance Mechanisms and Simple Mutation on Non-stationary Problems. In: Eiben, A.E., Bäck, T., Schoenauer, M., Schwefel, H.-P. (eds.) PPSN 1998. LNCS, vol. 1498, pp. 139–148. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  46. Lis, J., Eiben, A.E.: A multi–sexual genetic algorithm for multiobjective optimization. In: Fukuda, T., Furuhashi, T. (eds.) Proc. 3rd IEEE Conference on Evolutionary Computation, pp. 59–64. IEEE Press, Piscataway (1996)

    Google Scholar 

  47. Lopez Cruz, I.L., Van Willigenburg, L.G., Van Straten, G.: Efficient Differential Evolution algorithms for multimodal optimal control problems. Applied Soft Computing 3, 97–122 (2003)

    Article  Google Scholar 

  48. McElwain, J.C., Punyasena, S.W.: Mass extinction events and the plant fossil record. Trends in Ecology & Evolution 22, 548–557 (2007)

    Article  Google Scholar 

  49. Meyers, L.A., Bull, J.J.: Fighting change with change: adaptive variation in an uncertain world. Trends in Ecology & Evolution 17, 551–557 (2002)

    Article  Google Scholar 

  50. Michalewicz, Z., Janikow, C.Z., Krawczyk, J.B.: A modified genetic algorithm for optimal control problems. Computers and Mathematics with Applications 23, 83–94 (1992)

    Article  MATH  Google Scholar 

  51. Morrison, R.W., De Jong, K.A.: Triggered hypermutation revisited. In: Zalzala, A., Fonseca, C., Kim, J.H., Smith, A., Yao, X. (eds.) Proc. Congress on Evolutionary Computation, IEEE CEC 2000, pp. 1025–1032. IEEE Press, Piscataway (2000)

    Google Scholar 

  52. Morrison, R.W.: Designing Evolutionary Algorithms for Dynamic Environments. Springer, Heidelberg (2004)

    MATH  Google Scholar 

  53. Paenke, I., Branke, J., Jin, Y.: On the influence of phenotype plasticity on genotype diversity. In: Fogel, D.B., Yao, X., Mendel, J., Omori, T. (eds.) Proc. IEEE Symposium on Foundations of Computational Intelligence, FOCI 2007, pp. 33–40. IEEE Press, Piscataway (2007)

    Chapter  Google Scholar 

  54. Paenke, I., Branke, J., Jin, Y.: Balancing population– and individual–level adaptation in changing environments. Adaptive Behavior 17, 153–174 (2009)

    Article  Google Scholar 

  55. Parter, M., Kashtan, N., Alon, U.: Facilitated variation: How evolution learns from past environments to generalize to new environments. PLoS Comput. Biol. 4(11), e1000206 (2008), doi:10.1371/journal.pcbi.1000206

    Article  Google Scholar 

  56. Pigliucci, M., Kaplan, J.M.: Making Sense of Evolution: The Conceptual Foundations of Evolutionary Biology. University of Chicago Press, Chicago (2006)

    Google Scholar 

  57. Rahnamayan, S., Tizhoosh, H.R., Salama, M.M.H.: A novel population initialization method for accelerating evolutionary algorithms. Computers & Mathematics with Applications 53, 1605–1614 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  58. Richter, H.: Behavior of Evolutionary Algorithms in Chaotically Changing Fitness Landscapes. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 111–120. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  59. Richter, H.: A study of dynamic severity in chaotic fitness landscapes. In: Corne, D. (ed.) Proc. 2005 IEEE Congress on Evolutionary Computation, pp. 2824–2831. IEEE Press, Piscataway (2005)

    Chapter  Google Scholar 

  60. Richter, H.: Evolutionary Optimization in Spatio–temporal Fitness Landscapes. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 1–10. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  61. Richter, H.: Coupled map lattices as spatio–temporal fitness functions: Landscape measures and evolutionary optimization. Physica D237, 167–186 (2008)

    Google Scholar 

  62. Richter, H.: Detecting change in dynamic fitness landscapes. In: Tyrrell, A. (ed.) Proc. Congress on Evolutionary Computation, IEEE CEC 2009, pp. 1613–1620. IEEE Press, Piscataway (2009)

    Chapter  Google Scholar 

  63. Richter, H.: Change detection in dynamic fitness landscapes: An immunological approach. In: Abraham, A., Carvalho, A., Herrera, F., Pai, V. (eds.) World Congress on Nature and Biologically Inspired Computing (NaBIC 2009), pp. 719–724. IEEE Research Publishing Services, Singapore (2009)

    Chapter  Google Scholar 

  64. Richter, H.: Evolutionary optimization and dynamic fitness landscapes: From reaction-diffusion systems to chaotic CML. In: Zelinka, I., Celikovsky, S., Richter, H., Chen, G. (eds.) Evolutionary Algorithms and Chaotic Systems, pp. 409–446. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  65. Richter, H., Yang, S.: Memory Based on Abstraction for Dynamic Fitness Functions. In: Giacobini, M., Brabazon, A., Cagnoni, S., Di Caro, G.A., Drechsler, R., Ekárt, A., Esparcia-Alcázar, A.I., Farooq, M., Fink, A., McCormack, J., O’Neill, M., Romero, J., Rothlauf, F., Squillero, G., Uyar, A.Ş., Yang, S., et al. (eds.) EvoWorkshops 2008. LNCS, vol. 4974, pp. 596–605. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  66. Richter, H., Yang, S.: Learning behavior in abstract memory schemes for dynamic optimization problems. Soft Computing 13, 1163–1173 (2009)

    Article  MATH  Google Scholar 

  67. Sahney, S., Benton, M.J.: Recovery from the most profound mass extinction of all time. Proc. of the Royal Society B275, 759–765 (2008)

    Article  Google Scholar 

  68. Seiffertt, J., Sanyal, S., Wunsch, D.C.: Hamilton–Jacobi–Bellman equations and approximate dynamic programming on time scales. IEEE Trans. Syst., Man, & Cybern. Part B: Cybern. 38, 918–923 (2008)

    Article  Google Scholar 

  69. Simões, A., Costa, E.: Variable-Size Memory Evolutionary Algorithm to Deal with Dynamic Environments. In: Giacobini, M., et al. (eds.) EvoWorkshops 2007. LNCS, vol. 4448, pp. 617–626. Springer, Heidelberg (2007)

    Google Scholar 

  70. Simões, A., Costa, E.: Evolutionary Algorithms for Dynamic Environments: Prediction Using Linear Regression and Markov Chains. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 306–315. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  71. Sontag, E.D.: A “universal” construction of Artstein’s theorem on nonlinear stabilization. Systems & Control Letters 13, 117–123 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  72. Stadler, B.M.R., Stadler, P.F., Wagner, G.P., Fontana, W.: The topology of the possible: Formal spaces underlying patterns of evolutionary change. J. Theor. Biol. 213, 241–274 (2001)

    Article  MathSciNet  Google Scholar 

  73. Tinós, R., Yang, S.: A self–organizing random immigrants genetic algorithm for dynamic optimization problems. Genet. Program. Evol. Mach. 8, 255–286 (2007)

    Article  Google Scholar 

  74. Tsinias, J.: Sufficient Lyapunov–like conditions for stabilization. Math. Control Signals Systems 2, 343–347 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  75. Uyar, A.Ş., Harmanci, A.E.: A new population based adaptive dominance change mechanism for diploid genetic algorithms in dynamic environments. Soft Computing 9, 803–815 (2005)

    Article  MATH  Google Scholar 

  76. Wagner, A.: Robustness and Evolvability in Living Systems. Princeton University Press, Princeton (2007)

    Google Scholar 

  77. Wang, F.Y., Zhang, H., Liu, D.: Adaptive dynamic programming: An introduction. IEEE Computational Intelligence Magazine 4, 39–47 (2009)

    Article  Google Scholar 

  78. Werbos, P.J.: A menu of designs for reinforcement learning over time. In: Miller, W.T., Sutton, R.S., Werbos, P.J. (eds.) Neural Networks for Control, pp. 67–95. MIT Press, Cambridge (1991)

    Google Scholar 

  79. Werbos, P.J.: Approximate dynamic programming for real–time control and neural modeling. In: White, D.A., Sofge, D.A. (eds.) Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, pp. 493–525. Van Nostrand Reinhold, New York (1992)

    Google Scholar 

  80. Yang, S.: Associative Memory Scheme for Genetic Algorithms in Dynamic Environments. In: Rothlauf, F., Branke, J., Cagnoni, S., Costa, E., Cotta, C., Drechsler, R., Lutton, E., Machado, P., Moore, J.H., Romero, J., Smith, G.D., Squillero, G., Takagi, H., et al. (eds.) EvoWorkshops 2006. LNCS, vol. 3907, pp. 788–799. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  81. Zhang, X.S.: Evolution and maintenance of the environmental component of the phenotypic variance: Benefit of plastic traits under changing environments. The American Naturalist 166, 569–580 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Information Technology, Department of Measurement Technology and Control Engineering, HTWK Leipzig University of Applied Sciences, D–04251, Leipzig, Germany

    Hendrik Richter

  2. Department of Information Systems and Computing, Brunel University, Uxbridge, Middlesex, UB8 3PH, United Kingdom

    Shengxiang Yang

Authors
  1. Hendrik Richter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shengxiang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hendrik Richter .

Editor information

Editors and Affiliations

  1. Technical University of Ostrava, 17. listopadu 15, Ostrava-Porub, 708 33, Czech Republic

    Ivan Zelinka

  2. , Faculty of Electrical Engineering and, Technical University of Ostrava, 17. listopadu 15, Ostrava-Poruba, 708 33, Czech Republic

    Václav Snášel

  3. Auburn, 98071, USA

    Ajith Abraham

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Richter, H., Yang, S. (2013). Dynamic Optimization Using Analytic and Evolutionary Approaches: A Comparative Review. In: Zelinka, I., Snášel, V., Abraham, A. (eds) Handbook of Optimization. Intelligent Systems Reference Library, vol 38. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30504-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-30504-7_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-30503-0

  • Online ISBN: 978-3-642-30504-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation