Skip to main content
Springer Nature Link
Log in

Novelty and Beyond: Towards Combined Motivation Models and Integrated Learning Architectures

  • Chapter
  • First Online:
Intrinsically Motivated Learning in Natural and Artificial Systems

Abstract

For future intrinsically motivated agents to combine multiple intrinsic motivation or behavioural components, there is a need to identify fundamental units of motivation models that can be reused and combined to produce more complex agents. This chapter reviews three existing models of intrinsic motivation, novelty, interest and competence-seeking motivation, that are based on the neural network framework of a real-time novelty detector. Four architectures are discussed that combine basic units of the intrinsic motivation functions in different ways. This chapter concludes with a discussion of future directions for combined motivation models and integrated learning architectures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Baldassarre, G.: What are intrinsic motivations? a biological perspective. In: proceedings of International Conference on Developmental Learning and Epigenetic Robotics, pp. 1–8. IEEE, New York (2011)

    Google Scholar 

  2. Baraldi, A., Alpaydin, E.: Simplified art: A new class of art algorithms. Technical report, International Computer Science Institute, Berkley (1998)

    Google Scholar 

  3. Barto, A., Singh, S., Chentanez, N.: Intrinsically motivated learning of hierarchical collections of skills. In: International Conference on Developmental Learning (2004)

    Google Scholar 

  4. Berlyne, D.: Conflict, Arousal and Curiosity. McGraw-Hill, New York (1960)

    Google Scholar 

  5. Berlyne, D.: Exploration and curiosity. Science 153, 25–33 (1966)

    Google Scholar 

  6. BLIPSystems: Blipnet technical white paper. BLIP Systems A/S, Aalborg (2007)

    Google Scholar 

  7. Chandola, V., Banerjee, A, Kumar, V.: Anomaly detection: A survey. ACM Comput. Surv. 41(3), 1–58 (2009)

    Article  Google Scholar 

  8. Deci, E., Ryan, R.: Intrinsic Motivation and Self-determination in Human Behaviour. Plenum, New York (1985)

    Book  Google Scholar 

  9. Fritzke, B.: Incremental learning of local linear mappings. In: The International Conference on Artificial Neural Networks, pp. 217–222 (1995)

    Google Scholar 

  10. Gero, J.: Creativity, emergence and evolution in design. In: Second International Roundtable Conference on Computational Models of Creative Design, pp. 1–28, Sydney (1992)

    Google Scholar 

  11. Heckhausen, J., Heckhausen, H.: Motivation and Action. Cambridge University Press, New York (2008)

    Book  Google Scholar 

  12. Hettich, S., Bay, S.: The uci kdd archive (1999). http://kdd.ics.uci.edu/databases/kddcup99/kddcup99 (Accessed March 2009)

  13. Hill, C., Amodeo, A., Joseph, J., Patel, H.: Nano- and microrobotics: How far is the reality. Expert Rev. 8(12), 1891–1897 (2008)

    Google Scholar 

  14. Kaplan, F., Oudeyer, P.-Y.: Intrinsically motivated machines. In: Lungarella, M., Iida, F.J.B., Pfeifer, R. (eds.) Fifty Years of Artificial Intelligence, pp. 304–315. Springer, Berlin (2007)

    Google Scholar 

  15. King, R., et al.: The automation of science. Science 324(5923), 85–89 (2009)

    Article  Google Scholar 

  16. Kohonen, T.: Self-organisation and Associative Memory. Springer, Berlin (1993)

    Google Scholar 

  17. Lee, W., Stolfo, S.: A framework for constructing features and models for intrusion detection systems. ACM Trans. Inform. Syst. Security 3(4), 227–261 (2001)

    Article  Google Scholar 

  18. Lenat, D.: AM: An artificial intelligence approach to discovery in mathematics. Ph.D. Thesis, Stanford University (1976)

    Google Scholar 

  19. MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: The Fifth Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297. University of California Press (1967)

    Google Scholar 

  20. Maher, M., Gero, J.: Agent models of 3d virtual worlds. In: ACADIA 2002: Thresholds, pp. 127–138. California State Polytechnic University (2002)

    Google Scholar 

  21. Maher, M.L., Merrick, K., Saunders, R.: Achieving Creative Behaviour Using Curious Learning Agents, AAAI Spring Symposium on Creative Intelligent Systems, March 26–28, Stanford University, pp. 40–46 (2008).

    Google Scholar 

  22. Marsland, S., Nehmzow, U., Shapiro, J.: A real-time novelty detector for a mobile robot. In: EUREL European Advanced Robotics Systems Masterclass and Conference (2000)

    Google Scholar 

  23. Merrick, K.: Designing toys that come alive: Curious robots for creative play. In: Seventh International Conference on Entertainment Computing (ICEC 2008), pp. 149–154. Carnegie Mellon University, Springer (2008)

    Google Scholar 

  24. Merrick, K.: A comparative study of value systems for self-motivated exploration and learning by robots. IEEE Trans. Auton. Mental Dev. 2(2), 119–131 (2010a). Special Issue on Active Learning and Intrinsically Motivated Exploration in Robots

    Google Scholar 

  25. Merrick, K.: Modeling behavior cycles as a value system for developmental robots. Adap. Behav. 18(3–4), 237–257 (2010b)

    Article  Google Scholar 

  26. Merrick, K., Maher, M.: Motivated Reinforcement Learning: Curious Characters for Multiuser Games. Springer, Berlin (2009)

    Book  Google Scholar 

  27. Merrick, K., Scully, T.: Modelling affordances for the control and evaluation of intrinsically motivated robots. In: Australian Conference on Robotics and Automation, CD. Sydney (2009)

    Google Scholar 

  28. Merrick, K., Shafi, K.: Agent models for self-motivated home-assistant bots. In: International Symposium on Computational Models for Life Sciences, pp. 131–150. AIP (2009)

    Google Scholar 

  29. Merrick, K., Shafi, K.: Achievement, affiliation and power: Motive profiles for artificial agents. Adap. Behav. 9(1), 40–62 (2011)

    Article  Google Scholar 

  30. Mirolli, M., Baldassarre, G.: Functions and mechanisms of intrinsic motivations: The knowledge versus competence distinction. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 49–72. Springer, Berlin (2012)

    Google Scholar 

  31. Nehmzow, U., Gatsoulis, Y., Kerr, E., Condell, J., Siddique, N.H., McGinnity, M.T.: Novelty detection as an intrinsic motivation for cumulative learning robots. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 185–207. Springer, Berlin (2012)

    Google Scholar 

  32. Oudeyer, P.-Y., Kaplan, F.: Intelligent adaptive curiosity: A source of self-development. In: Fourth International Workshop on Epigenetic Robotics, pp. 127–130. Lund University, Lund (2004)

    Google Scholar 

  33. Oudeyer, P.-Y., Kaplan, F.: What is intrinsic motivation? a typology of computational approaches. Front. Neurorobot. 1(6) (2007) (online, openaccess)

    Google Scholar 

  34. Oudeyer, P.-Y., Kaplan, F., Hafner, V.: Intrinsic motivation systems for autonomous mental development. IEEE Trans. Evol. Comput. 11(2), 265–286 (2007)

    Article  Google Scholar 

  35. Saunders, R.: Curious design agents and artificial creativity. Ph.D. Thesis, University of Sydney, Sydney (2001)

    Google Scholar 

  36. Schembri, M., Mirolli, M., Baldassarre, G.: Evolution and learning in an intrinsically motivated reinforcement learning robot. In: Advances in Artificial Life. Proceedings of the 9th European Conference on Artificial Life (ECAL2007), Lecture Notes in Artificial Intelligence, vol. 4648, pp. 294–333. Springer, Berlin (2007)

    Google Scholar 

  37. Schmidhuber, J.: Curious model building control systems. In: International Joint Conference on Artificial Neural Networks, pp. 1458–1463. IEEE, Singapore (1991)

    Google Scholar 

  38. Schmidhuber, J.: Formal theory of creativity, fun and intrinsic motivation (1990-2010): IEEE Trans. Auton. Mental Dev. 2(3), 230–247 (2010)

    Article  Google Scholar 

  39. Shafi, K., Merrick, K.: A curious agent for network anomaly detection. In: The Tenth International Conference on Autonomous Agents and Multiagent Systems, pp. 1075–1076 (2011)

    Google Scholar 

  40. Stanley, J.: Computer simulation of a model of habituation. Nature 261, 146–148 (1976)

    Article  Google Scholar 

  41. Steels, L.: Self-organising vocabularies. Vrije Universiteit, Artificial Intelligence Laboratory, Brussels (1996)

    Google Scholar 

  42. Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT, Cambridge (2000)

    MATH  Google Scholar 

  43. Watkins, C., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)

    MATH  Google Scholar 

  44. Weiser, M.: The computer for the 21st century. Sci. Am. 265(3), 94–104 (1991)

    Article  Google Scholar 

  45. White, R.: Motivation reconsidered: The concept of competence. Psychol. Rev. 66, 297–333 (1959)

    Article  Google Scholar 

  46. Wundt, W.: Principles of Physiological Psychology. Macmillan, New York (1910)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Engineering and Information Technology, University of New South Wales, Australian Defence Force Academy, Northcott Drive, Canberra, ACT, Australia

    Kathryn E. Merrick

Authors
  1. Kathryn E. Merrick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kathryn E. Merrick .

Editor information

Editors and Affiliations

  1. Istituto di Scienze e Tecnologie della Cognizione Consiglio Nazionale delle Ricerche, Rome, Italy

    Gianluca Baldassarre  & Marco Mirolli  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Merrick, K.E. (2013). Novelty and Beyond: Towards Combined Motivation Models and Integrated Learning Architectures. In: Baldassarre, G., Mirolli, M. (eds) Intrinsically Motivated Learning in Natural and Artificial Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32375-1_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32375-1_9

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32374-4

  • Online ISBN: 978-3-642-32375-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation