Skip to main content
SpringerLink
Log in
Book cover

International Conference on Artificial General Intelligence

AGI 2015: Artificial General Intelligence pp 394–403Cite as

On the Limits of Recursively Self-Improving AGI

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9205))

Abstract

Self-improving software has been a goal of computer scientists since the founding of the field of Artificial Intelligence. In this work we analyze limits on computation which might restrict recursive self-improvement. We also introduce Convergence Theory which aims to predict general behavior of RSI systems.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Yampolskiy, R.V., Construction of an NP Problem with an Exponential Lower Bound (2011). Arxiv preprint arXiv:1111.0305

    Google Scholar 

  2. Yonck, R.: Toward a Standard Metric of Machine Intelligence. World Future Review 4(2), 61–70 (2012)

    Article  Google Scholar 

  3. Bremermann, H.J.: Quantum noise and information. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability (1967)

    Google Scholar 

  4. Bekenstein, J.D.: Information in the holographic universe. Scientific American 289(2), 58–65 (2003)

    Article  MathSciNet  Google Scholar 

  5. Lloyd, S.: Ultimate Physical Limits to Computation. Nature 406, 1047–1054 (2000)

    Article  Google Scholar 

  6. Sandberg, A.: The physics of information processing superobjects: daily life among the Jupiter brains. Journal of Evolution and Technology 5(1), 1–34 (1999)

    Google Scholar 

  7. Aaronson, S.: Guest column: NP-complete problems and physical reality. ACM Sigact News 36(1), 30–52 (2005)

    Article  Google Scholar 

  8. Shannon, C.E.: A Mathematical Theory of Communication. Bell Systems Technical Journal 27(3), 379–423 (1948)

    Article  MathSciNet  MATH  Google Scholar 

  9. Krauss, L.M., Starkman, G.D.: Universal limits on computation (2004). arXiv preprint astro-ph/0404510

    Google Scholar 

  10. Fox, D.: The limits of intelligence. Scientific American 305(1), 36–43 (2011)

    Article  Google Scholar 

  11. Einstein, A.: Does the inertia of a body depend upon its energy-content? Annalen der Physik 18, 639–641 (1905)

    Article  Google Scholar 

  12. Wheeler, J.A.: Information, Physics, Quantum: The Search for Links. Univ. of Texas (1990)

    Google Scholar 

  13. Schaeffer, J., et al.: Checkers is Solved. Science 317(5844), 1518–1522 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  14. Mahoney, M.: Is there a model for RSI?. In: SL4, June 20, 2008. http://www.sl4.org/archive/0806/19028.html

  15. Turing, A.: On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society 2(42), 230–265 (1936)

    MathSciNet  Google Scholar 

  16. Wiedermann, J.: A Computability Argument Against Superintelligence. Cognitive Computation 4(3), 236–245 (2012)

    Article  Google Scholar 

  17. Wiedermann, J.: Is There Something Beyond AI? Frequently Emerging, but Seldom Answered Questions about Artificial Super-Intelligence, p. 76. Artificial Dreams, Beyond AI

    Google Scholar 

  18. Mahoney, M.: A Model for Recursively Self Improving Programs (2010). http://mattmahoney.net/rsi.pdf

  19. Yudkowsky, E., Intelligence Explosion Microeconomics. In: MIRI Technical Report. www.intelligence.org/files/IEM.pdf

  20. Rice, H.G.: Classes of recursively enumerable sets and their decision problems. Transactions of the American Mathematical Society 74(2), 358–366 (1953)

    Article  MathSciNet  MATH  Google Scholar 

  21. Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1(1), 67–82 (1997)

    Article  Google Scholar 

  22. Melkikh, A.V.: The No Free Lunch Theorem and hypothesis of instinctive animal behavior. Artificial Intelligence Research 3(4), p43 (2014)

    Article  Google Scholar 

  23. de Garis, H.: The 21st. Century Artilect: Moral Dilemmas Concerning the Ultra Intelligent Machine. Revue Internationale de Philosophie 44(172), 131–138 (1990)

    Google Scholar 

  24. Yudkowsky, E., Herreshoff, M.: Tiling agents for self-modifying AI, and the Löbian obstacle. In: MIRI Technical Report (2013)

    Google Scholar 

  25. Fallenstein, B., Soares, N.: Problems of self-reference in self-improving space-time embedded intelligence. In: MIRI Technical Report (2014)

    Google Scholar 

  26. Yudkowsky, E.: The Procrastination Paradox (Brief technical note). In: MIRI Technical Report (2014). https://intelligence.org/files/ProcrastinationParadox.pdf

  27. Bolander, T.: Logical theories for agent introspection. Comp. Science 70(5), 2002 (2003)

    Google Scholar 

  28. Orseau, L.: Ring, M.: Self-modification and mortality in artificial agents. In: 4th international conference on Artificial general intelligence, pp. 1–10. Mount. View, CA. (2011)

    Google Scholar 

  29. Yampolskiy, R.V.: Utility Function Security in Artificially Intelligent Agents. Journal of Experimental and Theoretical Artificial Intelligence (JETAI), 1–17 (2014)

    Google Scholar 

  30. Chalmers, D.: The Singularity: A Philosophical Analysis. Journal of Consciousness Studies 17, 7–65 (2010)

    Google Scholar 

  31. Yampolskiy, R.V.: Artificial intelligence safety engineering: Why machine ethics is a wrong approach. In: Philosophy and Theory of Artificial Intelligence, pp. 389–396, Springer (2013)

    Google Scholar 

  32. Wolfram, S.: A New Kind of Science. Wolfram Media, Inc., May 14, 2002

    Google Scholar 

  33. Yampolskiy, R.V.: Computing Partial Solutions to Difficult AI Problems. In: Midwest Artificial Intelligence and Cognitive Science Conference, p. 90 (2012)

    Google Scholar 

  34. Böckenhauer, H.-J., Hromkovič, J., Mömke, T., Widmayer, P.: On the hardness of reoptimization. In: Geffert, V., Karhumäki, J., Bertoni, A., Preneel, B., Návrat, P., Bieliková, M. (eds.) SOFSEM 2008. LNCS, vol. 4910, pp. 50–65. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  35. Ausiello, G., Escoffier, B., Monnot, J., Paschos, V.T.: Reoptimization of minimum and maximum traveling salesman’s tours. In: Arge, L., Freivalds, R. (eds.) SWAT 2006. LNCS, vol. 4059, pp. 196–207. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  36. Archetti, C., Bertazzi, L., Speranza, M.G.: Reoptimizing the traveling salesman problem. Networks 42(3), 154–159 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  37. Ausiello, G., Bonifaci, V., Escoffier, B.: Complexity and approximation in reoptimization. Imperial College Press/World Scientific (2011)

    Google Scholar 

  38. Loosemore, R., Goertzel, B.: Why an intelligence explosion is probable. In: Singularity Hypotheses, pp. 83–98. Springer (2012)

    Google Scholar 

  39. Shahaf, D., Amir, E.: Towards a theory of AI completeness. In: 8th International Symposium on Logical Formalizations of Commonsense Reasoning. California, March 26–28, 2007

    Google Scholar 

  40. Yampolskiy, R.V.: Turing test as a defining feature of AI-completeness. In: Yang, X.-S. (ed.) Artificial Intelligence, Evolutionary Computing and Metaheuristics. SCI, vol. 427, pp. 3–17. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  41. Yampolskiy, R.V.: AI-complete, AI-hard, or AI-easy–classification of problems in AI. In: The 23rd Midwest Artificial Intelligence and Cognitive Science Conference, OH, USA (2012)

    Google Scholar 

  42. Yudkowsky, E.S.: General Intelligence and Seed AI (2001). http://singinst.org/ourresearch/publications/GISAI/

  43. Yampolskiy, R.V.: Efficiency Theory: a Unifying Theory for Information, Computation and Intelligence. J. of Discrete Math. Sciences & Cryptography 16(4–5), 259–277 (2013)

    Article  Google Scholar 

  44. Yampolskiy, R.V.: AI-Complete CAPTCHAs as Zero Knowledge Proofs of Access to an Artificially Intelligent System. ISRN Artificial Intelligence 271878 (2011)

    Google Scholar 

  45. Turing, A.M.: On Computable Numbers, with an Application to the Entscheidungsproblem. Proceedings of the London Mathematical Society 42, 230–265 (1936)

    MathSciNet  Google Scholar 

  46. Bostrom, N.: Superintelligence: Paths, dangers, strategies. Oxford University Press (2014)

    Google Scholar 

  47. Hall, J.S.: Engineering utopia. Frontiers in AI and Applications 171, 460 (2008)

    Google Scholar 

  48. Hutter, M.: Universal algorithmic intelligence: A mathematical top→ down approach. In: Artificial general intelligence, pp. 227–290. Springer (2007)

    Google Scholar 

  49. Kolmogorov, A.N.: Three Approaches to the Quantitative Definition of Information. Problems Inform. Transmission 1(1), 1–7 (1965)

    MathSciNet  Google Scholar 

  50. Yampolskiy, R.V.: The Universe of Minds (2014). arXiv:1410.0369

    Google Scholar 

  51. Yudkowsky, E.: Levels of organization in general intelligence. In: Artificial general intelligence, pp. 389–501. Springer (2007)

    Google Scholar 

  52. Bostrom, N.: What is a Singleton? Linguistic and Philosophical Invest. 5(2), 48–54 (2006)

    Google Scholar 

  53. Yudkowsky, E.: Timeless decision theory. The Singularity Institute, San Francisco (2010)

    Google Scholar 

  54. LessWrong: Acausal Trade, September 29, 2014. http://wiki.lesswrong.com/wiki/Acausal_trade

  55. Yudkowsky, E.S.: Coherent Extrapolated Volition. Singularity Institute for Artificial Intelligence, May 2004. http://singinst.org/upload/CEV.html

  56. Yudkowsky, E.: Recursive Self-Improvement. In: Less Wrong, December 1, 2008. http://lesswrong.com/lw/we/recursive_selfimprovement/, September 29, 2014

  57. Hutter, M.: Can Intelligence Explode? J. of Consciousness Studies 19(1–2), 1–2 (2012)

    Google Scholar 

  58. Yampolskiy, R.V.: Analysis of types of self-improving software. In: The Eighth Conference on Artificial General Intelligence, Berlin, Germany, July 22–25, 2015

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering and Computer Science, Speed School of Engineering, University of Louisville, Louisville, USA

    Roman V. Yampolskiy

Authors
  1. Roman V. Yampolskiy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roman V. Yampolskiy .

Editor information

Editors and Affiliations

  1. Reykjavik University, Reykjavik, Iceland

    Jordi Bieger

  2. Hong Kong Polytechnic University, Hong Kong, Hong Kong SAR

    Ben Goertzel

  3. Mechanics and Optics, Saint Petersburg State University of Information Technologies, St. Petersburg, Russia

    Alexey Potapov

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Yampolskiy, R.V. (2015). On the Limits of Recursively Self-Improving AGI. In: Bieger, J., Goertzel, B., Potapov, A. (eds) Artificial General Intelligence. AGI 2015. Lecture Notes in Computer Science(), vol 9205. Springer, Cham. https://doi.org/10.1007/978-3-319-21365-1_40

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-21365-1_40

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-21364-4

  • Online ISBN: 978-3-319-21365-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation