Skip to main content
Springer Nature Link
Log in

Learning to Predict Variable-Delay Rewards and Its Role in Autonomous Developmental Robotics

  • Conference paper
  • First Online:
Bio-Inspired Applications of Connectionism (IWANN 2001)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2085))

Included in the following conference series:

  • 445 Accesses

Abstract

Researchers in the new field of “developmental robotics” propose to provide robots with so-called developmental programs. Similar to the development of human infants, robots might use those programs to interact with humans and their environment for extended periods of time, and become smarter autonomously. In this paper we show how a neural network model developed by neuroscientists can be used by an autonomous robot to learn by trial-and-error when considering rewards delivered at arbitrary times, as would be the case of developmental robots interacting with humans in the real world.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. A. Billard. Learning motor skills by imitation: a biologically inspired robotic model. Cybernetics and Systems, 32(1-2), 2001 (in press).

    Google Scholar 

  2. K. Doya. What are the computations in the cerebellum, the basal ganglia, and the cerebral cortex. Neural Networks, 12:961–974, 1999.

    Article  Google Scholar 

  3. K.J. Lang and G.E. Hinton. A time-delay neural network architecture for speech recognition. Technical Report CMU-DS-88-152, Dept. of Computer Science, Carnegie Mellon University, Pittsburgh, PA, December 1988.

    Google Scholar 

  4. F. Mondada, E. Franzi, and P. Ienne. Mobile robot miniaturization: A tool for investigating in control algorithms. In Proceedings of the Third International Symposium on Experimental Robotics, Kyoto, Japan, 1993.

    Google Scholar 

  5. A. Pérez-Uribe. Structure-Adaptable Digital Neural Networks, chapter 6.A Neurocontroller Architecture for Autonomous Robots, pages 95–116. Swiss Federal Institute of Technology-Lausanne, Ph.D Thesis 2052, 1999.

    Google Scholar 

  6. A. Pérez-Uribe. Using a time-delay actor-critic neural architecture with dopamine-like reinforcement signal for learning in autonomous robots. In S. Wermter, J. Austin, and D. Willshaw, editors, Emergent Neural Computational Architectures based on Neuroscience. Springer-Verlag, 2001 (to appear).

    Google Scholar 

  7. S. Schaal. Is imitation learning the route to humanoid robots?. Trends in Cognitive Sciences, 3(6):233–242, 1999.

    Article  Google Scholar 

  8. W. Schultz, P. Dayan, and P. Read Montague. A Neural Substrate of Prediction and Reward. Science, 275:1593–1599, 14 March 1997.

    Article  Google Scholar 

  9. M. Sipper, E. Sanchez, D. Mange, M. Tomassini, A. Pérez-Uribe, and A. Stauffer. A Phylogenetic, Ontogenetic, and Epigenetic View of Bio-Inspired Hardware Systems. IEEE Transactions on Evolutionary Computation, 1(1):83–97, April 1997.

    Article  Google Scholar 

  10. R.E. Suri and W. Schultz. A Neural Network Model With Dopamine-Like Reinforcement Signal That Learns a Spatial Delayed Responde Task. Neuroscience, 91(3):871–890, 1999.

    Article  Google Scholar 

  11. R.E. Suri and W. Schultz. Internal Model Reproduces Anticipatory Neural Activity. (available at http://www.snl.salk.edu/suri), July 1999.

  12. R.S. Sutton and A.G. Barto. Reinforcement Learning: An Introduction. The MIT Press, 1998.

    Google Scholar 

  13. J. Weng, W.S. Hwang, Y. Zhang, C. Yang, and R. Smith. Developmental Humanoids: Humanoids that Develop Skills Automatically. In Proceedings the first IEEE-RAS International Conference on Humanoid Robots, Cambridge MA, September 7-8 2000.

    Google Scholar 

  14. J. Weng, J. McClelland, A. Pentland, O. Sporns, I. Stockman, M. Sur, and E. Thelen. Autonomous Mental Development by Robots and Animals. Science, 291(5504):599–600, 2000.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Parallelism and Artificial Intelligence Group Department of Informatics, University of Fribourg, Switzerland

    Andrés Pérez-Uribe* & Michèle Courant

Authors
  1. Andrés Pérez-Uribe*
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michèle Courant
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Departamento de Inteligencia Artificial, Universidad Nacional de Educación a Distancia, Senda del Rey, s/n., 28040, Madrid, Spain

    José Mira

  2. Departamento de Arquitectura y Tecnología de Computadores, Universidad de Granada, Campus Fuentenueva, 18071, Granada, Spain

    Alberto Prieto

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Pérez-Uribe*, A., Courant, M. (2001). Learning to Predict Variable-Delay Rewards and Its Role in Autonomous Developmental Robotics. In: Mira, J., Prieto, A. (eds) Bio-Inspired Applications of Connectionism. IWANN 2001. Lecture Notes in Computer Science, vol 2085. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45723-2_59

Download citation

  • DOI: https://doi.org/10.1007/3-540-45723-2_59

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-42237-2

  • Online ISBN: 978-3-540-45723-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation