Skip to main content
SpringerLink
Log in

Evolutionary Computation and AI Safety

Research Problems Impeding Routine and Safe Real-World Application of Evolution

  • Chapter
  • First Online:
Genetic Programming Theory and Practice XVII

Part of the book series: Genetic and Evolutionary Computation ((GEVO))

Abstract

Recent developments in artificial intelligence and machine learning have spurred interest in the growing field of AI safety, which studies how to prevent human-harming accidents when deploying AI systems. This paper thus explores the intersection of AI safety with evolutionary computation, to show how safety issues arise in evolutionary computation and how understanding from evolutionary computational and biological evolution can inform the broader study of AI safety.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Google photos labeled black people ‘gorillas’. https://www.usatoday.com/story/tech/2015/07/01/google-apologizes-after-photos-identify-black-people-as-gorillas/29567465/. Accessed: 2019-05-01

  2. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety. arXiv preprint arXiv:1606.06565 (2016)

    Google Scholar 

  3. Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robotics and Autonomous Systems 57(5), 469–483 (2009)

    Article  Google Scholar 

  4. Armstrong, S., Levinstein, B.: Low impact artificial intelligences. arXiv preprint arXiv:1705.10720 (2017)

    Google Scholar 

  5. Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming — An Introduction. Morgan Kaufmann, San Francisco (1998)

    Book  MATH  Google Scholar 

  6. Branke, J., Schmeck, H.: Designing evolutionary algorithms for dynamic optimization problems. In: Advances in Evolutionary Computing, pp. 239–262. Springer (2003)

    Google Scholar 

  7. Buss, D.: Evolutionary psychology: The new science of the mind. Psychology Press (2015)

    Google Scholar 

  8. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: A survey. ACM computing Surveys (CSUR) 41(3), 15 (2009)

    Google Scholar 

  9. Christiano, P., Shlegeris, B., Amodei, D.: Supervising strong learners by amplifying weak experts. arXiv preprint arXiv:1810.08575 (2018)

    Google Scholar 

  10. Cobbe, K., Klimov, O., Hesse, C., Kim, T., Schulman, J.: Quantifying generalization in reinforcement learning. arXiv preprint arXiv:1812.02341 (2018)

    Google Scholar 

  11. Everitt, T., Lea, G., Hutter, M.: Agi safety literature review. arXiv preprint arXiv:1805.01109 (2018)

    Google Scholar 

  12. Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., Song, D.: Robust physical-world attacks on deep learning models. arXiv preprint arXiv:1707.08945 (2017)

    Google Scholar 

  13. Eysenbach, B., Gu, S., Ibarz, J., Levine, S.: Leave no trace: Learning to reset for safe and autonomous reinforcement learning. arXiv preprint arXiv:1711.06782 (2017)

    Google Scholar 

  14. Gaier, A., Asteroth, A., Mouret, J.B.: Data-efficient design exploration through surrogate-assisted illumination. Evolutionary Computation 26(3), 381–410 (2018)

    Article  Google Scholar 

  15. Garcıa, J., Fernández, F.: A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research 16(1), 1437–1480 (2015)

    MathSciNet  MATH  Google Scholar 

  16. Goldsby, H.J., Cheng, B.H.: Automatically discovering properties that specify the latent behavior of UML models. In: International Conference on Model Driven Engineering Languages and Systems, pp. 316–330. Springer (2010)

    Google Scholar 

  17. Goodhart, C.A.: Problems of monetary management: The UK experience. In: Monetary Theory and Practice, pp. 91–121. Springer (1984)

    Google Scholar 

  18. Hadfield-Menell, D., Hadfield, G.K.: Incomplete contracting and ai alignment. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 417–422. ACM (2019)

    Google Scholar 

  19. Hadfield-Menell, D., Milli, S., Abbeel, P., Russell, S.J., Dragan, A.: Inverse reward design. In: Advances in Neural Information Processing Systems, pp. 6765–6774 (2017)

    Google Scholar 

  20. Irving, G., Christiano, P., Amodei, D.: AI safety via debate. arXiv preprint arXiv:1805.00899 (2018)

    Google Scholar 

  21. Jakobi, N., Husbands, P., Harvey, I.: Noise and the reality gap: The use of simulation in evolutionary robotics. In: European Conference on Artificial Life, pp. 704–720. Springer (1995)

    Google Scholar 

  22. Jin, Y.: Surrogate-assisted evolutionary computation: Recent advances and future challenges. Swarm and Evolutionary Computation 1(2), 61–70 (2011)

    Article  Google Scholar 

  23. Justesen, N., Torrado, R.R., Bontrager, P., Khalifa, A., Togelius, J., Risi, S.: Procedural level generation improves generality of deep reinforcement learning. arXiv preprint arXiv:1806.10729 (2018)

    Google Scholar 

  24. Kahn, G., Villaflor, A., Pong, V., Abbeel, P., Levine, S.: Uncertainty-aware reinforcement learning for collision avoidance. arXiv preprint arXiv:1702.01182 (2017)

    Google Scholar 

  25. Karpov, I.V., Valsalam, V.K., Miikkulainen, R.: Human-assisted neuroevolution through shaping, advice and examples. In: Proceedings of the 13th annual Conference on Genetic and evolutionary computation, pp. 371–378. ACM (2011)

    Google Scholar 

  26. Kashtan, N., Noor, E., Alon, U.: Varying environments can speed up evolution. Proceedings of the National Academy of Sciences 104(34), 13,711–13,716 (2007)

    Article  Google Scholar 

  27. Kirschner, M., Gerhart, J.: Evolvability. Proceedings of the National Academy of Sciences 95(15), 8420–8427 (1998)

    Article  Google Scholar 

  28. Koos, S., Mouret, J.B., Doncieux, S.: The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Transactions on Evolutionary Computation 17(1), 122–145 (2013)

    Article  Google Scholar 

  29. Koza, J.R.: Genetic programming: On the programming of computers by means of natural selection. MIT press (1992)

    Google Scholar 

  30. Koza, J.R.: Genetic programming II: Automatic discovery of reusable subprograms. Cambridge, MA, USA 13(8), 32 (1994)

    Google Scholar 

  31. Kurakin, A., Goodfellow, I., Bengio, S.: Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533 (2016)

    Google Scholar 

  32. Langton, C.G.: Artificial life: An overview. Mit Press (1997)

    Google Scholar 

  33. Lehman, J., Chen, J., Clune, J., Stanley, K.O.: ES is more than just a traditional finite-difference approximator. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 450–457. ACM (2018)

    Google Scholar 

  34. Lehman, J., Chen, J., Clune, J., Stanley, K.O.: Safe mutations for deep and recurrent neural networks through output gradients. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 117–124. ACM (2018)

    Google Scholar 

  35. Lehman, J., Clune, J., Misevic, D., Adami, C., Altenberg, L., Beaulieu, J., Bentley, P.J., Bernard, S., Beslon, G., Bryson, D.M., et al.: The surprising creativity of digital evolution: A collection of anecdotes from the evolutionary computation and artificial life research communities. arXiv preprint arXiv:1803.03453 (2018)

    Google Scholar 

  36. Lehman, J., Miikkulainen, R.: Neuroevolution. Scholarpedia 8(6), 30,977 (2013)

    Article  Google Scholar 

  37. Lehman, J., Stanley, K.O.: Evolving a diversity of virtual creatures through novelty search and local competition. In: Proceedings of the 13th annual Conference on Genetic and Evolutionary Computation, pp. 211–218. ACM (2011)

    Google Scholar 

  38. Lehman, J., Stanley, K.O.: Improving evolvability through novelty search and self-adaptation. In: 2011 IEEE Congress of Evolutionary Computation (CEC), pp. 2693–2700. IEEE (2011)

    Google Scholar 

  39. Leike, J., Krueger, D., Everitt, T., Martic, M., Maini, V., Legg, S.: Scalable agent alignment via reward modeling: A research direction. arXiv preprint arXiv:1811.07871 (2018)

    Google Scholar 

  40. Leike, J., Martic, M., Krakovna, V., Ortega, P.A., Everitt, T., Lefrancq, A., Orseau, L., Legg, S.: AI safety gridworlds. arXiv preprint arXiv:1711.09883 (2017)

    Google Scholar 

  41. Lenat, D.B.: Cyc: A large-scale investment in knowledge infrastructure. Communications of the ACM 38(11), 33–38 (1995)

    Article  Google Scholar 

  42. Lewis, M.A., Fagg, A.H., Solidum, A.: Genetic programming approach to the construction of a neural network for control of a walking robot. In: Proceedings 1992 IEEE International Conference on Robotics and Automation, pp. 2618–2623. IEEE (1992)

    Google Scholar 

  43. Lipton, Z.C., Azizzadenesheli, K., Kumar, A., Li, L., Gao, J., Deng, L.: Combating reinforcement learning’s sisyphean curse with intrinsic fear. arXiv preprint arXiv:1611.01211 (2016)

    Google Scholar 

  44. Markou, M., Singh, S.: Novelty detection: A review - part 1: Statistical approaches. Signal Processing 83(12), 2481–2497 (2003)

    Article  MATH  Google Scholar 

  45. Moldovan, T.M., Abbeel, P.: Safe exploration in Markov decision processes. arXiv preprint arXiv:1205.4810 (2012)

    Google Scholar 

  46. Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 427–436 (2015)

    Google Scholar 

  47. Nolfi, S., Floreano, D., Floreano, D.D.: Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. MIT press (2000)

    Google Scholar 

  48. Ong, C.S., Huang, J.J., Tzeng, G.H.: Building credit scoring models using genetic programming. Expert Systems with Applications 29(1), 41–47 (2005)

    Article  Google Scholar 

  49. Pollack, J.B., Lipson, H., Ficici, S., Funes, P., Hornby, G., Watson, R.A.: Evolutionary techniques in physical robotics. In: International Conference on Evolvable Systems, pp. 175–186. Springer (2000)

    Google Scholar 

  50. Pugh, J.K., Soros, L.B., Stanley, K.O.: Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI 3, 40 (2016)

    Article  Google Scholar 

  51. Richter, H.: Detecting change in dynamic fitness landscapes. In: 2009 IEEE Congress on Evolutionary Computation, pp. 1613–1620. IEEE (2009)

    Google Scholar 

  52. Rocha, M., Cortez, P., Neves, J.: Evolution of neural networks for classification and regression. Neurocomputing 70(16–18), 2809–2816 (2007)

    Article  Google Scholar 

  53. Ross, S.A.: The economic theory of agency: The principal’s problem. The American Economic Review 63(2), 134–139 (1973)

    Google Scholar 

  54. Saunders, W., Sastry, G., Stuhlmueller, A., Evans, O.: Trial without error: Towards safe reinforcement learning via human intervention. In: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, pp. 2067–2069. International Foundation for Autonomous Agents and Multiagent Systems (2018)

    Google Scholar 

  55. Secretan, J., Beato, N., D Ambrosio, D.B., Rodriguez, A., Campbell, A., Stanley, K.O.: Picbreeder: Evolving pictures collaboratively online. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1759–1768. ACM (2008)

    Google Scholar 

  56. Shaker, N., Togelius, J., Nelson, M.J.: Procedural content generation in games. Springer (2016)

    Google Scholar 

  57. Sims, K.: Evolving virtual creatures. In: Proceedings of the 21st annual conference on Computer graphics and interactive techniques, pp. 15–22. ACM (1994)

    Google Scholar 

  58. Soltoggio, A., Bullinaria, J.A., Mattiussi, C., Dürr, P., Floreano, D.: Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios. In: Proceedings of the 11th international Conference on Artificial Life (Alife XI), CONF, pp. 569–576. MIT Press (2008)

    Google Scholar 

  59. Soltoggio, A., Stanley, K.O., Risi, S.: Born to learn: the inspiration, progress, and future of evolved plastic artificial neural networks. Neural Networks 108, 48–67 (2018)

    Article  Google Scholar 

  60. Standish, R.K.: Open-ended artificial evolution. International Journal of Computational Intelligence and Applications 3(02), 167–175 (2003)

    Article  Google Scholar 

  61. Stanley, K.O., Bryant, B.D., Miikkulainen, R.: Real-time neuroevolution in the NERO video game. IEEE Transactions on Evolutionary Computation 9(6), 653–668 (2005)

    Article  Google Scholar 

  62. Stanley, K.O., Lehman, J., Soros, L.: Open-endedness: The last grand challenge you’ve never heard of. While open-endedness could be a force for discovering intelligence, it could also be a component of AI itself (2017)

    Google Scholar 

  63. Sutton, R.S.: Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In: Machine Learning Proceedings 1990, pp. 216–224. Elsevier (1990)

    Google Scholar 

  64. Takagi, H.: Interactive evolutionary computation: Fusion of the capabilities of EC optimization and human evaluation. Proceedings of the IEEE 89(9), 1275–1296 (2001)

    Article  Google Scholar 

  65. Taylor, J.: Quantilizers: A safer alternative to maximizers for limited optimization. In: Workshops at the Thirtieth AAAI Conference on Artificial Intelligence (2016)

    Google Scholar 

  66. Taylor, T., Bedau, M., Channon, A., Ackley, D., Banzhaf, W., Beslon, G., Dolson, E., Froese, T., Hickinbotham, S., Ikegami, T., et al.: Open-ended evolution: Perspectives from the OEE workshop in York. Artificial Life 22(3), 408–423 (2016)

    Article  Google Scholar 

  67. Vann, M.G.: Of rats, rice, and race: The great Hanoi rat massacre, an episode in French colonial history. French Colonial History 4(1), 191–203 (2003)

    Article  Google Scholar 

  68. Wagner, G.P., Altenberg, L.: Perspective: Complex adaptations and the evolution of evolvability. Evolution 50(3), 967–976 (1996)

    Article  Google Scholar 

  69. Wang, R., Lehman, J., Clune, J., Stanley, K.O.: Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions. arXiv preprint arXiv:1901.01753 (2019)

    Google Scholar 

  70. Watson, R.A., Ficici, S.G., Pollack, J.B.: Embodied evolution: Distributing an evolutionary algorithm in a population of robots. Robotics and Autonomous Systems 39(1), 1–18 (2002)

    Article  Google Scholar 

  71. Whittaker, M., Crawford, K., Dobbe, R., Fried, G., Kaziunas, E., Mathur, V., West, S.M., Richardson, R., Schultz, J., Schwartz, O.: AI now report 2018. AI Now Institute at New York University (2018)

    Google Scholar 

  72. Wierstra, D., Schaul, T., Peters, J., Schmidhuber, J.: Natural evolution strategies. In: 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), pp. 3381–3387. IEEE (2008)

    Google Scholar 

  73. Woolley, B.G., Stanley, K.O.: A novel human-computer collaboration: Combining novelty search with interactive evolution. In: Proceedings of the 2014 annual Conference on Genetic and Evolutionary Computation, pp. 233–240. ACM (2014)

    Google Scholar 

  74. Yao, X.: Evolving artificial neural networks. Proceedings of the IEEE 87(9), 1423–1447 (1999)

    Article  Google Scholar 

  75. Yudkowsky, E.: Coherent extrapolated volition. Singularity Institute for Artificial Intelligence (2004)

    Google Scholar 

  76. Zafar, M.B., Valera, I., Gomez Rodriguez, M., Gummadi, K.P.: Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In: Proceedings of the 26th International Conference on World Wide Web, pp. 1171–1180. International World Wide Web Conferences Steering Committee (2017)

    Google Scholar 

  77. Zhang, C., Vinyals, O., Munos, R., Bengio, S.: A study on overfitting in deep reinforcement learning. arXiv preprint arXiv:1804.06893 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Uber AI, San Francisco, CA, USA

    Joel Lehman

Authors
  1. Joel Lehman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joel Lehman .

Editor information

Editors and Affiliations

  1. Computer Science and Engineering, John R. Koza Chair, Michigan State University, East Lansing, MI, USA

    Wolfgang Banzhaf

  2. BEACON Center, Michigan State University, East Lansing, MI, USA

    Erik Goodman

  3. Department of Computer Science and Engineering, Michigan State University, Okemos, MI, USA

    Leigh Sheneman

  4. Depto Ingenieria en Electronic Electrica Tecnológico Nacional de México/ IT, Tijuana, Baja California, Mexico

    Leonardo Trujillo

  5. Evolution Enterprises, Ann Arbor, MI, USA

    Bill Worzel

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Lehman, J. (2020). Evolutionary Computation and AI Safety. In: Banzhaf, W., Goodman, E., Sheneman, L., Trujillo, L., Worzel, B. (eds) Genetic Programming Theory and Practice XVII. Genetic and Evolutionary Computation. Springer, Cham. https://doi.org/10.1007/978-3-030-39958-0_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-39958-0_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-39957-3

  • Online ISBN: 978-3-030-39958-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation