Skip to main content
SpringerLink
Log in

Robot life-long task learning from human demonstrations: a Bayesian approach

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Programming a robot to act intelligently is a challenging endeavor beyond the skill level of most people. Trained roboticists generally program robots for a single purpose. Enabling robots to be programmed by non-experts and to perform multiple tasks are both open challenges in robotics. This paper presents a framework that allows life-long robot task learning from demonstrations. To make that possible, the paper introduces a task representation based on influence diagrams, and a method to transfer knowledge between similar tasks. A novel approach to influence diagram learning is presented along with a demonstration method that allows non-experts to teach tasks to the robot in an intuitive manner. The results from three user studies validate that the approach enables both a simulated and a physical robot to learn complex tasks from a variety of teachers, refining those tasks during on-line performance, successfully completing the tasks in different environments, and transferring knowledge from one task to another.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5), 469–483.

    Article  Google Scholar 

  • Atkeson, C., & Schaal, S. (1997). Robot learning from demonstration. In Proceedings of the Fourteenth International Conference on Machine Learning.

  • Bentivegna, D. (2004). Learning from observation using primitives. PhD thesis, College of Computing, Georgia Institute of Technology, Atlanta, GA.

  • Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008). Robot programming by demonstration, Chapter 59. Cambridge: MIT Press.

    Google Scholar 

  • Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. Washington, D.C.: National Academy Press.

    Google Scholar 

  • Brunner, C., Peynot, T., & Underwood, J. (2009). Towards discrimination of challenging conditions for ugvs with visual and infrared sensors’. In Proceedings of 2009 Australasian Conference on Robotics and Automation (ACRA), Sydney, Australia, Decemeber 2009.

  • Chernova, S., & M. Veloso. (2007). Confidence-based policy learning from demonstration using gaussian mixture models. In International Conference on Autonomous Agents and Multiagent Systems.

  • Chernova, S., & Veloso, M. (2008). Teaching collaborative multi-robot tasks through demonstration. In IEEE-RAS International Conference on Humanoid Robots, Daejeon, Korea.

  • Churchill, W., & Newman, P. (2012). Practice makes perfect? managing and leveraging visual experiences for lifelong navigation. In Proc. IEEE International Conference on Robotics and Automation (ICRA2012), Minnesota, USA.

  • Ciocarlie, M. (2012). Ros table top perception, 8 2012. http://www.ros.org/wiki/tabletop_object_perception.

  • Dillmann, R., Kaiser, M. & Ude, A. (1995). Acquisition of elementary robot skills from human demonstration. In International Symposium on Intelligent Robotic Systems.

  • Endsley, M. R. (1988). Design and evaluation for situation awareness enhancement. In Proceedings of the Human Factors Society 32nd Annual Meeting, Santa Monica, CA: Human Factors Society.

  • Garland A., & Lesh, N. (2003). Learning hierarchical taskmodels by demonstration. Technical Report Technical Report TR2003-01, Mitsubishi Electric Research Laboratories.

  • Gold, K. (2009) An information pipeline model of human-robot interaction. In Proceedings of the 4th ACM/IEEE international conference on Human robot interaction (pp. 85–92). New York, NY, USA: ACM. ISBN 978-1-60558-404-1. doi:10.1145/1514095.1514113.

  • Grollman, D. & Jenkins, O. (2007a). Dogged learning for robots. In Proceedings of the IEEE International Conference on Robotics and Automation.

  • Grollman, D. H. & Jenkins, O. C. (2007b). Learning robot soccer skills from demon-stration. In International Conference on Development and Learning (pp 276–281), London, UK.

  • Grollman, D. H. & Jenkins, O. C. (2008). Sparse incremental learning for interactive robot control policy estimation. In International Conference on Robotics and Automation (pp 3315–3320) Pasadena, CA, USA.

  • Hentschel, M., & Wagner, B. (2011). An adaptive memory model for long-term navigation of autonomous mobile robots. Journal of Robotics, 2011, 506245. doi:10.1155/2011/506245.

  • Hersch, M., Guenter, F., Calinon, S., & Billard, A. (2008). Dynamical system modulation for robot learning via kinesthetic demonstrations. IEEE Transactions on Robotics, 24(6), 1463–1467.

    Article  Google Scholar 

  • Ijspeert, J. A., Nakanishi, J. & Schaal, S. (2002). Movement imitation with nonlinear dynamical systems in humanoid robots. In international conference on robotics and automation (icra2002).

  • Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237–285.

    Google Scholar 

  • Koenig, N. (2012). Robot life-long task learning from human demonstrations: A bayesian approach. Cres-12-002. Los Angeles, CA: University of Southern California.

    Google Scholar 

  • Koenig, N. & Howard, A. (2004). Design and use paradigms for gazebo, an open-source multi-robot simulator. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp 2149–2154) Sendai, Japan.

  • Koenig, N., Takayama, L., & Matarić, M. (2010). Communication and knowledge sharing in human-robot interaction. Neural Networks, 23, 1104–1112.

    Article  Google Scholar 

  • Kuniyoshi, Y., Inaba, M., & Inoue, H. (1994). Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transaction on Robotics and Automation, 10, 799–822.

    Article  Google Scholar 

  • Matarić, M. J. (1997). Behavior-based control: Examples from navigation, learning, and group behavior. Journal of Experimental and Theoretical Artificial Intelligence, 9(2), 323–336.

    Article  Google Scholar 

  • Mohseni-Kabir, A., Rich, C., Chernova, S., Sidner, C. L., & Miller, D. (2015). Interactive hierarchical task learning from a single demonstration. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction HRI ’15 (pp 205–212) New York, NY: ACM. ISBN 978-1-4503-2883-8. doi:10.1145/2696454.2696474. http://doi.acm.org/10.1145/2696454.2696474.

  • Myers, B. A. (1990). Creating user interfaces using programming by example, visual programming, and constraints. ACM Transactions on Programming Languages and Systems, 12(2), 143–177.

    Article  Google Scholar 

  • Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., & Kawato., M. (2004). Learning from demonstration and adaptation of biped locomotion. Robotics and autonomous systems, 1(2–3), 79–91.

    Article  Google Scholar 

  • Nicolescu, M., & Matarić, M. (2003). Methods for robot task learning: Demonstrations, generalization and practice. In Proceedings of the Second International Joint Conference on Autonomous Agents and Multi-Agent Systems.

  • Pardowitz, M., & Dillmann, R. (2007). Towards life-long learning in household robots: The piagetian approach. In IEEE 6th International Conference on Development and Learning (pp 88–93).

  • Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009). Learning and generalization of motor skills by learning from demonstration. In international conference on robotics and automation.

  • Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Francisco, CA: Morgan Kaufmann.

    MATH  Google Scholar 

  • Peters, J., Vijayakumar, S., & Schaal S. (2003). Reinforcement learning for humanoid robotics. In IEEE-RAS International Conference on Humanoid Robots (Humanoids2003).

  • Pollard, N., & Hodgins, J. K. (2002). Generalizing demonstrated manipulation tasks. In Workshop on the Algorithmic Foundations of Robotics.

  • Pomerleau, D. (1991). Efficient training of artificial neural networks for autonomous navigation. Neural Computation, 3(1), 88–97.

    Article  Google Scholar 

  • Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., & Ng, A. Y. (2009). Ros: An open-source robot operating system. In ICRA Workshop on Open Source Software.

  • Rosenstein, M., & Barto, A. (2004). Supervised actor-critic reinforcement learning, chapter 14. New York, NY, USA: Wiley.

    Google Scholar 

  • Russell, S. (1998). Learning agents for uncertain environments. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory.

  • Schaal, S. (1999). Is imitation learning the route to humanoid robotics? Trends in Cognitive Sciences, 3(6), 233–242.

    Article  Google Scholar 

  • Schaal, S. Peters, J., Nakanishi, J., & Ijspeert, A. (2003). Control, planning, learning, and imitation with dynamic movement primitives. In Workshop on bilateral paradigms on humans and humanoids, IEEE International Conference on Intelligent Robots and Systems (IROS 2003).

  • Schaal, S., Ijspeert, A., & A. Billard. (2004). Computational approaches to motor learning by imitation (pp. 199–218). Number 1431. Oxford University Press.

  • Shachter, R. D. (1988). Probabilistic inference and influence diagrams. Operations Research, 36, 589–604.

    Article  MATH  Google Scholar 

  • Sutton, R. (1998). Reinforcement learning. Cambridge, MA: MIT Press.

    Google Scholar 

  • Thomaz, A., & Breazeal, C. (2006a). Reinforcement learning with human teachers: Evidence of feedback and guidance with implications for learning performance. In Proceedings of the 21st National Conference on Artificial Intelligence.

  • Thomaz, A. L., & Breazeal, C. (2006b). Reinforcement learning with human teachers: Understanding how people want to teach robots. In Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication

  • Thrun, S., & Mitchell, T. (1995). Lifelong robot learning. Robotics and Autonomous Systems, 15, 25–46.

    Article  Google Scholar 

  • Vijayakumar, S., & Schaal, S. (2000). Locally weighted projection regression: An \(o(n)\) algorithm for incremental real time learning in high dimensional space. In Proceedings of the 17th International Conference on Machine Learning.

Download references

Acknowledgments

This work was supported by NSF grant for “SHB: Small: Socially Assistive Human-Machine Interaction for Improved Compliance and Health Outcomes” IIS-1117279; ONR MURI grant “ANTIDOTE: Adaptive Networks for Threat and Intrusian Detection Or TErmination” N00014-09-1-1031; and ONR DURIP Grant “Acquisition of a Personal Robotic Platform for DoD-Sponsored Research in human–robot Interaction, Motor Control, and Perception” N000141210729.

Author information

Authors and Affiliations

  1. Open Source Robotics Foundation, Mountain View, USA

    Nathan Koenig

  2. University of Southern California, Los Angeles, USA

    Maja J. Matarić

Authors
  1. Nathan Koenig
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maja J. Matarić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nathan Koenig.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koenig, N., Matarić, M.J. Robot life-long task learning from human demonstrations: a Bayesian approach. Auton Robot 41, 1173–1188 (2017). https://doi.org/10.1007/s10514-016-9601-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-016-9601-1

Keywords

Search

Navigation