Skip to main content
Springer Nature Link
Log in

Observation of Unbounded Novelty in Evolutionary Algorithms is Unknowable

  • Conference paper
  • First Online:
Artificial Intelligence and Soft Computing (ICAISC 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10841))

Included in the following conference series:

  • 2286 Accesses

Abstract

Open ended evolution seeks computational structures whereby creation of unbounded diversity and novelty are possible. However, research has run into a problem known as the “novelty plateau” where further creation of novelty is not observed. Using standard algorithmic information theory and Chaitin’s Incompleteness Theorem, we prove no algorithm can detect unlimited novelty. Therefore observation of unbounded novelty in computer evolutionary programs is nonalgorithmic and, in this sense, unknowable.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Some authors use the notation “\(\overset{+}{=}\)” in lieu of “\(\underset{c}{=} \)” [31, 34].

  2. 2.

    Even though i is not prefix free (i.e. it will halt once \(b^*\) is executed, leaving \(a^*_{b^*}\) unread), the program that is run on the Turing machine \(\mathcal {U}\) is still prefix free because i is appended to \(p_c\), so the full execution on \(\mathcal {U}\) is \(\mathcal {U}(p_ci)\). Since \(p_c\) is prefix free, then so is \(p_ci\), as it will only halt once the entire string is read.

  3. 3.

    While Kolmogorov complexity is an exact metric, and has to account for both meaningful structure and random noise in the population, the Kolmogorov sufficient statistic can be used to measure just the meaningful structure in the population.

References

  1. Mitchell, M., Forrest, S.: Genetic algorithms and artificial life. Artif. life 1(3), 267–289 (1994)

    Article  Google Scholar 

  2. Huneman, P.: Determinism, predictability and open-ended evolution: lessons from computational emergence. Synthese 185(2), 195–214 (2012)

    Article  MathSciNet  Google Scholar 

  3. Komosinski, M., Rotaru-Varga, A.: From directed to open-ended evolution in a complex simulation model. Artif. Life 7, 293–299 (2000)

    Google Scholar 

  4. Sayama, H.: Seeking open-ended evolution in swarm chemistry. In: 2011 IEEE Symposium on Artificial Life (ALIFE), pp. 186–193. IEEE (2011)

    Google Scholar 

  5. Li, J., Storie, J., Clune, J.: Encouraging creative thinking in robots improves their ability to solve challenging problems. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 193–200. ACM (2014)

    Google Scholar 

  6. Soros, L., Stanley, K.O.: Identifying necessary conditions for open-ended evolution through the artificial life world of chromaria. Artif. life 14, 793–800 (2014)

    Google Scholar 

  7. Basener, W.F.: Exploring the concept of open-ended evolution. In: Biological Information: New Perspectives, pp. 87–104. World Scientific (2012)

    Google Scholar 

  8. Bedau, M.A., McCaskill, J.S., Packard, N.H., Rasmussen, S., Adami, C., Green, D.G., Ikegami, T., Kaneko, K., Ray, T.S.: Open problems in artificial life. Artif. life 6(4), 363–376 (2000)

    Article  Google Scholar 

  9. Ruiz-Mirazo, K., Peretó, J., Moreno, A.: A universal definition of life: autonomy and open-ended evolution. Orig. Life Evol. Biosph. 34(3), 323–346 (2004)

    Article  Google Scholar 

  10. Ruiz-Mirazo, K., Umerez, J., Moreno, A.: Enabling conditions for open-ended evolution. Biol. Philos. 23(1), 67–85 (2008)

    Article  Google Scholar 

  11. Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm Intelligence: from Natural to Artificial Systems, vol. 1. Oxford University Press, Oxford (1999)

    MATH  Google Scholar 

  12. Ewert, W., Marks, R.J., Thompson, B.B., Yu, A.: Evolutionary inversion of swarm emergence using disjunctive combs control. IEEE Trans. Syst. Man Cybern. Syst. 43(5), 1063–1076 (2013)

    Article  Google Scholar 

  13. Roach, J., Ewert, W., Marks, R.J., Thompson, B.B.: Unexpected emergent behaviors from elementary swarms. In: 2013 45th Southeastern Symposium on System theory (SSST), pp. 41–50. IEEE (2013)

    Google Scholar 

  14. Roach, J.H., Marks, R.J., Thompson, B.B.: Recovery from sensor failure in an evolving multiobjective swarm. IEEE Trans. Syst. Man Cybern. Syst. 45(1), 170–174 (2015)

    Article  Google Scholar 

  15. Taylor, T.: Exploring the concept of open-ended evolution. In: Proceedings of the 13th International Conference on Artificial life, pp. 540–541 (2012)

    Google Scholar 

  16. Jakobi, N.: Encoding scheme issues for open-ended artificial evolution. In: Voigt, H.-M., Ebeling, W., Rechenberg, I., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 52–61. Springer, Heidelberg (1996). https://doi.org/10.1007/3-540-61723-X_969

    Chapter  Google Scholar 

  17. Channon, A.: Three evolvability requirements for open-ended evolution. In: Artificial Life VII Workshop Proceedings, Portland, OR, pp. 39–40 (2000)

    Google Scholar 

  18. Mueller, I.: Euclid’s elements and the axiomatic method. Br. J. Philos. Sci. 20(4), 289–309 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  19. Shenoy, P.P., Shafer, G.: Axioms for probability and belief-function propagation. In: Yager, R.R., Liu, L. (eds.) Classic Works of the Dempster-Shafer Theory of Belief Functions. STUDFUZZ, vol. 219, pp. 499–528. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-44792-4_20

    Chapter  Google Scholar 

  20. Chaitin, G.J.: Algorithmic information theory. IBM J. Res. Dev. 21(4), 350–359 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  21. Raatikainen, P.: On interpreting Chaitin’s incompleteness theorem. J. Philos. Log. 27(6), 569–586 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  22. Chaitin, G.J.: Information, Randomness & Incompleteness: Papers on Algorithmic Information Theory, vol. 8. World Scientific, Singapore (1990)

    Book  MATH  Google Scholar 

  23. Grünwald, P.D., Vitányi, P.M., et al.: Algorithmic information theory. In: Handbook of the Philosophy of Information, pp. 281–320 (2008)

    Chapter  Google Scholar 

  24. Seibt, P.: Algorithmic Information Theory. Springer, Heidelberg (2006). https://doi.org/10.1007/978-3-540-33219-0

    Book  MATH  Google Scholar 

  25. Van Lambalgen, M.: Algorithmic information theory. J. Symb. Log. 54(4), 1389–1400 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  26. Chaitin, G.: Proving Darwin: Making Biology Mathematical. Vintage, New York (2012)

    MATH  Google Scholar 

  27. Chaitin, G.J.: Toward a mathematical definition of life. In: Information, Randomness & Incompleteness: Papers on Algorithmic Information Theory, pp. 86–104. World Scientific (1987)

    Google Scholar 

  28. Gecow, A.: The purposeful information. On the difference between natural and artificial life. Dialogue Univers. 18(11/12), 191–206 (2008)

    Article  Google Scholar 

  29. Pattee, H.H.: Artificial life needs a real epistemology. In: Morán, F., Moreno, A., Merelo, J.J., Chacón, P. (eds.) ECAL 1995. LNCS, vol. 929, pp. 21–38. Springer, Heidelberg (1995). https://doi.org/10.1007/3-540-59496-5_286

    Chapter  Google Scholar 

  30. Chaitin, G.J.: The Unknowable. Springer Science & Business Media, Heidelberg (1999)

    MATH  Google Scholar 

  31. Bennett, C.H., Gács, P., Li, M., Vitanyi, P., Zurek, W.H.: Information distance. arXiv preprint arXiv:1006.3520 (2010)

  32. Calude, C.S.: Algorithmic randomness, quantum physics, and incompleteness. In: Margenstern, M. (ed.) MCU 2004. LNCS, vol. 3354, pp. 1–17. Springer, Heidelberg (2005). https://doi.org/10.1007/978-3-540-31834-7_1

    Chapter  Google Scholar 

  33. Ming, L., Vitányi, P.M.: Kolmogorov complexity and its applications. Algorithms Complex. 1, 187 (2014)

    MATH  Google Scholar 

  34. Vitányi, P.M., Li, M.: Minimum description length induction, Bayesianism, and Kolmogorov complexity. IEEE Trans. Inf. Theory 46(2), 446–464 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  35. Wallace, C.S., Dowe, D.L.: Minimum message length and Kolmogorov complexity. Comput. J. 42(4), 270–283 (1999)

    Article  MATH  Google Scholar 

  36. Muller, G.B., Wagner, G.P.: Novelty in evolution: restructuring the concept. Ann. Rev. Ecol. Syst. 22(1), 229–256 (1991)

    Article  Google Scholar 

  37. Pigliucci, M.: What, if anything, is an evolutionary novelty? Philos. Sci. 75(5), 887–898 (2008)

    Article  Google Scholar 

  38. Li, X., Croft, W.B.: An information-pattern-based approach to novelty detection. Inf. Process. Manag. 44(3), 1159–1188 (2008)

    Article  Google Scholar 

  39. Zhao, L., Zhang, M., Ma, S.: The nature of novelty detection. Inf. Retr. 9(5), 521–541 (2006)

    Article  Google Scholar 

  40. Kowaliw, T., Dorin, A., McCormack, J.: An empirical exploration of a definition of creative novelty for generative art. In: Korb, K., Randall, M., Hendtlass, T. (eds.) ACAL 2009. LNCS (LNAI), vol. 5865, pp. 1–10. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10427-5_1

    Chapter  Google Scholar 

  41. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 15 (2009)

    Article  Google Scholar 

  42. Venkatasubramanian, V., Rengaswamy, R., Yin, K., Kavuri, S.N.: A review of process fault detection and diagnosis: part I: quantitative model-based methods. Comput. Chem. Eng. 27(3), 293–311 (2003)

    Article  Google Scholar 

  43. Hodge, V., Austin, J.: A survey of outlier detection methodologies. Artif. Intell. Rev. 22(2), 85–126 (2004)

    Article  MATH  Google Scholar 

  44. Pimentel, M.A., Clifton, D.A., Clifton, L., Tarassenko, L.: A review of novelty detection. Sig. Process. 99, 215–249 (2014)

    Article  Google Scholar 

  45. Reed, R., Marks, R.J.: Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks. MIT Press, Cambridge (1999)

    Book  Google Scholar 

  46. Thompson, B.B., Marks, R.J., Choi, J.J., El-Sharkawi, M.A., Huang, M.Y., Bunje, C.: Implicit learning in autoencoder novelty assessment. In: 2002 Proceedings of the 2002 International Joint Conference on Neural Networks, IJCNN 2002, vol. 3, pp. 2878–2883. IEEE (2002)

    Google Scholar 

  47. Lehman, J., Stanley, K.O.: Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19(2), 189–223 (2011)

    Article  Google Scholar 

  48. Mouret, J.B.: Novelty-based multiobjectivization. In: Doncieux, S., Bredèche, N., Mouret, J.B. (eds.) New Horizons in Evolutionary Robotics. SCI, vol. 341, pp. 139–154. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-18272-3_10

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Baylor University, Waco, TX, 76706, USA

    Eric Holloway & Robert Marks

Authors
  1. Eric Holloway
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert Marks
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eric Holloway .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  3. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  4. University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  5. AGH University of Science and Technology, Kraków, Poland

    Ryszard Tadeusiewicz

  6. University of Louisville, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Holloway, E., Marks, R. (2018). Observation of Unbounded Novelty in Evolutionary Algorithms is Unknowable. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2018. Lecture Notes in Computer Science(), vol 10841. Springer, Cham. https://doi.org/10.1007/978-3-319-91253-0_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91253-0_37

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91252-3

  • Online ISBN: 978-3-319-91253-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics