Skip to main content
SpringerLink
Log in

Learning Hierarchical Skills from Observation

  • Conference paper
  • First Online:
Discovery Science (DS 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2534))

Included in the following conference series:

Abstract

This paper addresses the problem of learning control skills from observation. In particular, we show how to infer a hierarchical, reactive program that reproduces and explains the observed actions of other agents, specifically the elements that are shared across multiple individuals. We infer these programs using a three-stage process that learns flat unordered rules, combines these rules into a classification hierarchy, and finally translates this structure into a hierarchical reactive program. The resulting program is concise and easy to understand, making it possible to view program induction as a practical technique for knowledge acquisition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Anderson, C., Draper, B., & Peterson, D. (2000). Behavioral cloning of student pilots with modular neural networks. Proceedings of the Seventeenth International Conference on Machine Learning (pp. 25–32). Stanford: Morgan Kaufmann.

    Google Scholar 

  • Brooks, R. (1986). A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, 2, 14–23.

    MathSciNet  Google Scholar 

  • Clark, P., Boswell, R. (1991). Rule induction with CN2: Some recent improvements. Proceedings of the European Working Session on Learning (pp. 151–163). Porto.

    Google Scholar 

  • Cypher, A. (Ed.). (1993). Watch what I do: Programming by demonstration. Cambridge, MA: MIT Press.

    Google Scholar 

  • Firby, J. (1989). Adaptive execution in complex dynamic worlds. PhD Thesis, Department of Computer Science, Yale University, New Haven, CT.

    Google Scholar 

  • George., M., Lansky, A., & Bessiere, P. (1985). A procedural logic. Proceedings of the Ninth International Joint Conference on Artificial Intelligence (pp. 516–523). Los Angeles: Morgan Kaufmann.

    Google Scholar 

  • Kaelbling, L. P., Littman, L. M., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237–285.

    Google Scholar 

  • Langley, P., & Stromsten, S. (2000). Learning context-free grammars with a simplicity bias. Proceedings of the Eleventh European Conference on Machine Learning (pp. 220–228). Barcelona: Springer-Verlag.

    Google Scholar 

  • Mitchell, T. M., Mahadevan, S., & Steinberg, L. (1985). Leap: A learning apprentice for VLSI design. Proceedings of the Ninth International Joint Conference on Artificial Intelligence (pp. 573–580). Los Angeles: Morgan Kaufmann.

    Google Scholar 

  • Moriarty, D. E., Schultz, A. C., & Grefenstette, J. J. (1999). Evolutionary algorithms for reinforcement learning. Journal of Artificial Intelligence Research, 11, 241–276.

    MATH  Google Scholar 

  • Nilsson, N. (1994). Teleoreactive programs for agent control. Journal of Artificial Intelligence Research, 1, 139–158.

    Google Scholar 

  • Pomerleau, D. (1991). Rapidly adapting artificial neural networks for autonomous navigation. Advances in Neural Information Processing Systems 3 (pp. 429–435). San Francisco: Morgan Kaufmann.

    Google Scholar 

  • Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Francisco: Morgan Kaufmann.

    Google Scholar 

  • Sammut, C. (1996). Automatic construction of reactive control systems using symbolic machine learning. Knowledge Engineering Review, 11, 27–42.

    Article  Google Scholar 

  • Schoppers, M. (1987). Universal plans for reactive robots in unpredictable environments. Proceedings of the Tenth International Joint Conference on Artificial Intelligence (pp. 1039–1046). Milan, Italy: Morgan Kaufmann.

    Google Scholar 

  • Segre, A. (1987). A learning apprentice system for mechanical assembly. Proceedings of the Third IEEE Conference on AI for Applications (pp. 112–117).

    Google Scholar 

  • Shapiro, D., Langley, P., & Shachter, R. (2001). Using background knowledge to speed reinforcement learning in physical agents. Proceedings of the Fifth International Conference on Autonomous Agents (pp. 254–261). Montreal: ACM Press.

    Google Scholar 

  • Shapiro, D. (2001). Value-driven agents. PhD thesis, Department of Management Science and Engineering, Stanford University, Stanford, CA.

    Google Scholar 

  • Shapiro, D., & Langley, P. (2002). Separating skills from preference: Using learning to program by reward. Proceedings of the Nineteenth International Conference on Machine Learning (pp. 570–577). Sydney: Morgan Kaufmann.

    Google Scholar 

  • Urbancic, T., & Bratko, I. (1994). Reconstructing human skill with machine learning. Proceedings of the Eleventh European Conference on Artificial Intelligence (pp. 498–502). Amsterdam: John Wiley.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computational Learning Laboratory Center for the Study of Language and Information, Stanford University, 94305-4115, Stanford, CA, USA

    Ryutaro Ichise, Daniel Shapiro & Pat Langley

  2. National Institute of Informatics, 101-8430, Tokyo, Japan

    Ryutaro Ichise

Authors
  1. Ryutaro Ichise
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Shapiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pat Langley
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Deutsches Forschungszentrum für Künstliche Intelligenz, Stuhlsatzenhausweg 3, 66123, Saarbrücken, Germany

    Steffen Lange

  2. National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda-ku, 101-8430, Tokyo, Japan

    Ken Satoh

  3. Department of Computer Science, University of Maryland, College Park, 20742, Maryland, MD, USA

    Carl H. Smith

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Ichise, R., Shapiro, D., Langley, P. (2002). Learning Hierarchical Skills from Observation. In: Lange, S., Satoh, K., Smith, C.H. (eds) Discovery Science. DS 2002. Lecture Notes in Computer Science, vol 2534. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36182-0_22

Download citation

  • DOI: https://doi.org/10.1007/3-540-36182-0_22

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-00188-1

  • Online ISBN: 978-3-540-36182-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation