Skip to main content
SpringerLink
Log in

Intrinsically Motivated Affordance Discovery and Modeling

  • Chapter
  • First Online:
Intrinsically Motivated Learning in Natural and Artificial Systems

Abstract

In this chapter, we argue that a single intrinsic motivation function for affordance discovery can guide long-term learning in robot systems. To these ends, we provide a novel definition of “affordance” as the latent potential for the closed-loop control of environmental stimuli perceived by sensors. Specifically, the proposed intrinsic motivation function rewards the discovery of such control affordances. We will demonstrate how this function has been used by a humanoid robot to learn a number of general purpose control skills that address many different tasks. These skills, for example, include strategies for finding, grasping, and placing simple objects. We further show how this same intrinsic reward function is used to direct the robot to build stable models of when the environment affords these skills.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Barto, A., Singh, S, Chentanez, N.: Intrinsically motivated learning of hierarchical collections of skills. In: Proceedings of the International Conference on Development and Learning (ICDL), La Jolla, CA, USA October (2004)

    Google Scholar 

  • Berry, D.A., Fristedt, B.: Bandit problems: Sequential allocation of experiments. In: Monographs on Statistics and Applied Probability, vol. viii + 275. Chapman & Hall, London (1985)

    Google Scholar 

  • Chemero, A.: An outline of a theory of affordances. Ecol. Psychol. 15(3), 181–195 (2003)

    Article  Google Scholar 

  • Connolly, C., Grupen, R.: Nonholonomic path planning using harmonic functions. Technical Report 94-50, University of Massachusetts, Amherst (1994)

    Google Scholar 

  • Detry, R., Popovic, M., Touati, Y., Baseski, E., Krüger, N, Piater, J.: Autonomous learning of object-specific grasp affordance densities. In: ICRA 2009 Workshop Approaches to Sensorimotor Learning on Humanoid Robots, Kobe, Japan (2009)

    Google Scholar 

  • Festinger, L.: A Theory of Cognitive Dissonance. Evanston, Row, Peterson (1957)

    Google Scholar 

  • Fitzpatrick, P., Metta, G., Natale, L., Rao, S., Sandini, G.: Learning about objects through action: Initial steps towards artificial cognition. In: IEEE International Conference on Robotics and Automation, Taipei, Taiwan (2003)

    Google Scholar 

  • Gibson, J.J.: The theory of affordances. In: Perceiving, Acting and Knowing: Toward an Ecological Psychology, pp. 67–82. Lawrence Erlbaum Associates, Hillsdale (1977)

    Google Scholar 

  • Harlow, H.: Learning and satiation of response in intrinsically motivated complex puzzle performances by monkeys. J. Comp. Psychol. Psychol. 43, 289–294 (1950)

    Google Scholar 

  • Hart, S.: The Development of Hierarchical Knowledge in Robot Systems. Ph.D. Thesis, Department of Computer Science, University of Massachusetts, Amherst (2009a)

    Google Scholar 

  • Hart, S.: An intrinsic reward for affordance exploration. In: Proceedings of the 8th IEEE International Conference on Development and Learning (ICDL), Shanghai, China (2009b)

    Google Scholar 

  • Hart, S., Grupen, R.: Natural task decomposition with intrinsic potential fields. In: Proceedings of the 2007 International Conference on Intelligent Robots and Systems (IROS), San Diego, CA (2007)

    Google Scholar 

  • Hart, S., Grupen, R.: Learning generalizable control programs. IEEE Trans. Auton. Mental Dev. 3(3), 216–231 (2011)

    Article  Google Scholar 

  • Hart, S., Sen, S., Grupen, R.: Generalization and transfer in robot control. In: 8th International Conference on Epigenetic Robotics (Epirob08), University of Sussex, Brighton, UK (2008a)

    Google Scholar 

  • Hart, S., Sen, S., Grupen, R.: Intrinsically motivated hierarchical manipulation. In: Proceedings of the 2008 IEEE Conference on Robots and Automation (ICRA), Pasadena, CA (2008b)

    Google Scholar 

  • Herrmann, J., Pawelzik, K., Geisel, T.: Learning predictive representations. Neurocomputing 32–33, 785–791 (2000)

    Article  Google Scholar 

  • Huang, X., Weng, J.: Novelty and reinforcement learning in the value system of developmental robots. In: Proceedings of the 2nd International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems. Edinburgh, Scotland (2002)

    Google Scholar 

  • Huber, M., Grupen, R.: A hybrid discrete dynamic systems approach to robot control. Technical Report 96-43, Department of Computer Science, University of Massachusetts Amherst, Amherst (1996)

    Google Scholar 

  • Huber, M., MacDonald, W, Grupen, R.: A control basis for multilegged walking. In: Proceedings of the Conference on Robotics and Automation. IEEE, Minneapolis, MN, USA (1996)

    Google Scholar 

  • Hull, C.: Principles of Behavior: An Introduction to Behavior Theory. Appleton-Century-Croft, New York (1943)

    Google Scholar 

  • Hunt, H.: Intrinsic motivation and its role in psychological development. Nebraska Symp. Motiv. 13, 189–282 (1965)

    Google Scholar 

  • Kagan, J.: Motives and development. J. Pers. Soc. Psychol. 22, 51–66 (1972)

    Article  Google Scholar 

  • Koditschek, D., Rimon, E.: Robot navigation functions on manifolds with boundary. Adv. Appl. Math. 11(4), 412–442 (1990)

    Article  MathSciNet  Google Scholar 

  • Kraft, D., Pugeault, N., Baseski, E., Popović, M., Kragic, D., Kalkan, S., Wörgötter, F, Krüger, N.: Birth of the object: Detection of objectness and extraction of shape through object action complexes. In: Proceedings of the 2008 International Conference on Cognitive Systems, Karlsruhe, DE (2008)

    Google Scholar 

  • Krüger, N., Geib, C., Piater, J., Petrick, R., Steedman, M., Wörgötter, F., Ude, A., Asfour, T., Kraft, D., Omrcen, D., Agostini, A, Dillmann, R.: Object-action complexes: Grounded abstractions of sensori-motor processes. Robot. Auton. Syst. 59(10), 740–757 (2011)

    Article  Google Scholar 

  • Krüger, N., Piater, J., Wörgötter, F., Geib, C., Petrick, R., Steedman, M., Ude, A., Asfour, T., Kraft, D., Omrcen, D., Hommel, B., Agostino, A., Kragic, D., Eklundh, J., Kruger, V, Dillmann, R.: A formal definition of object action complexes and examples at different levels of the process hierarchy. http://www.paco-plus.org (2009)

  • Modayil, J., Kupiers, B.: Autonomous development of a grounded object ontology by a learning robot. In: Proceedings of the Twenty-Second Conference on Artificial Intelligence (AAAI-07), Vancouver, British Columbia (2007)

    Google Scholar 

  • Montgomery, K.: The role of exploratory drive in learning. J. Comp. Psychol. Psychol. 47, 60–64 (1954)

    Google Scholar 

  • Nakamura, Y.: Advanced Robotics: Redundancy and Optimization. Addison-Wesley, Reading (1991)

    Google Scholar 

  • Oudeyer, P., Kaplan, F.: What is intrinsic motivation? a typology of computational approaches. Front. Neurorobot. 1(2) (2007)

    Google Scholar 

  • Oudeyer, P., Kaplan, F, Hafner, V.V.: Intrinsic motivation systems for autonomous mental development. IEEE Trans. Evol. Comput. 11, 265–286 (2007)

    Article  Google Scholar 

  • Piaget, J.: The Origins of Intelligence in Childhood. International Universities Press, New York (1952)

    Book  Google Scholar 

  • Platt, R., Fagg, A.H, Grupen, R.A.: Null space grasp control: Theory and experiments. IEEE Trans. Robot. Autom. 26, 282–295 (2010)

    Article  Google Scholar 

  • Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  • Rimon, E., Koditschek, D.: Exact robot navigation using artificial potential functions. IEEE Trans. Robot. Autom. 8(5), 501–518 (1992)

    Article  Google Scholar 

  • Şahin, E., Çakmak, M., Doǧar, M., Uǧur, E, Üçoluk, G.: To afford or not to afford: A formalization of affordances toward affordance-based robot control. Adap. Behav. 4(15), 447–472 (2007)

    Article  Google Scholar 

  • Schmidhuber, J.: Adaptive curiosity and adaptive confidence. Technical Report FKI-149-91, Institut fur Informatik, Technische Universitat Munchen (1991)

    Google Scholar 

  • Stoytchev, A.: Toward learning the binding affordances of objects: A behavior-grounded approach. In: Proceedings of the AAAI Spring Symposium on Developmental Robotics. Stanford University, Stanford, CA (2005)

    Google Scholar 

  • Sutton, R., Barto, A.: Reinforcement Learning. MIT, Cambridge (1998)

    MATH  Google Scholar 

  • Uǧur, E., Oztop, E, Şahin, E.: Learning object affordances for planning. In: ICRA 2009 Workshop Approaches to Sensorimotor Learning on Humanoid Robots, Kobe, Japan (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Manufacturing Systems Research Laboratory, General Motors R&D, 30500 Mound Road, Warren, MI, 48090, USA

    Stephen Hart

  2. Department of Computer Science, University of Massachusetts Amherst, 140 Governors Drive, Amherst, MA, 01003, USA

    Roderic Grupen

Authors
  1. Stephen Hart
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roderic Grupen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephen Hart .

Editor information

Editors and Affiliations

  1. Istituto di Scienze e Tecnologie della Cognizione Consiglio Nazionale delle Ricerche, Rome, Italy

    Gianluca Baldassarre  & Marco Mirolli  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Hart, S., Grupen, R. (2013). Intrinsically Motivated Affordance Discovery and Modeling. In: Baldassarre, G., Mirolli, M. (eds) Intrinsically Motivated Learning in Natural and Artificial Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32375-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32375-1_12

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32374-4

  • Online ISBN: 978-3-642-32375-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation