Skip to main content
SpringerLink
Log in
Book cover

International Conference on Social Robotics

ICSR 2016: Social Robotics pp 11–21Cite as

Adaptive Robot Assisted Therapy Using Interactive Reinforcement Learning

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9979))

Abstract

In this paper, we present an interactive learning and adaptation framework that facilitates the adaptation of an interactive agent to a new user. We argue that Interactive Reinforcement Learning methods can be utilized and integrated to the adaptation mechanism, enabling the agent to refine its learned policy in order to cope with different users. We illustrate our framework with a use case in the domain of Robot Assisted Therapy. We present our results of the learning and adaptation experiments against different simulated users, showing the motivation of our work and discussing future directions towards the definition and implementation of our proposed framework.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://www.choosemuse.com/.

References

  1. Andrade, K.d.O., Fernandes, G., Caurin, G.A., et al.: Dynamic player modelling in serious games applied to rehabilitation robotics. In: Robotics Symposium and Robocontrol, pp. 211–216. IEEE (2014)

    Google Scholar 

  2. Bellemare, M.G., Naddaf, Y., Veness, J., Bowling, M.: The arcade learning environment: an evaluation platform for general agents. J. Artif. Intell. Res. 47, 253–279 (2012)

    Google Scholar 

  3. Broekens, J.: Emotion and reinforcement: affective facial expressions facilitate robot learning. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) Artifical Intelligence for Human Computing. LNCS (LNAI), vol. 4451, pp. 113–132. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  4. Chanel, G., Rebetez, C., Bétrancourt, M., Pun, T.: Boredom, engagement and anxiety as indicators for adaptation to difficulty in games. In: Proceedings of the 12th International Conference on Entertainment and Media in the Ubiquitous Era, pp. 13–17. ACM (2008)

    Google Scholar 

  5. Chao, C., Cakmak, M., Thomaz, A.L.: Transparent active learning for robots. In: ACM/IEEE International Conference on Human-Robot Interaction, pp. 317–324. IEEE (2010)

    Google Scholar 

  6. Chernova, S., Veloso, M.: Interactive policy learning through confidence-based autonomy. J. Artif. Intell. Res. 34(1), 1 (2009)

    MathSciNet  MATH  Google Scholar 

  7. Chi, M., VanLehn, K., Litman, D., Jordan, P.: An evaluation of pedagogical tutorial tactics for a natural language tutoring system: a reinforcement learning approach. Int. J. Artif. Intell. Educ. 21(1–2), 83–113 (2011)

    Google Scholar 

  8. Cruz, F., Twiefel, J., Magg, S., Weber, C., Wermter, S.: Interactive reinforcement learning through speech guidance in a domestic scenario. In: International Joint Conference on Neural Networks, pp. 1–8. IEEE (2015)

    Google Scholar 

  9. Cuayáhuitl, H., van Otterlo, M., Dethlefs, N., et al.: Machine learning for interactive systems and robots: a brief introduction. In: Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication, pp. 19–28. ACM (2013)

    Google Scholar 

  10. Gallina, P., Bellotto, N., Di Luca, M.: Progressive co-adaptation in human-machine interaction. In: Informatics in Control, Automation and Robotics. IEEE (2015)

    Google Scholar 

  11. Giullian, N., et al.: Detailed requirements for robots in autism therapy. In: Proceedings of SMC 2010, pp. 2595–2602. IEEE (2010)

    Google Scholar 

  12. Goodrich, M., Colton, M., Brinton, B., Fujiki, M., Atherton, J., Robinson, L., Ricks, D., Maxfield, M., Acerson, A.: Incorporating a robot into an autism therapy team. IEEE Life Sciences (2012)

    Google Scholar 

  13. McCullagh, P., et al.: Assessment of task engagement using brain computer interface technology. In: Workshop Proceedings of the 11th International Conference on Intelligent Environments, vol. 19. IOS Press (2015)

    Google Scholar 

  14. Knox, W.B., Stone, P.: Reinforcement learning from simultaneous human and MDP reward. In: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, vol. 1, pp. 475–482 (2012)

    Google Scholar 

  15. Libin, A., Cohen-Mansfield, J.: Therapeutic robocat for nursing home residents with dementia: preliminary inquiry. Am. J. Alzheimer’s Dis. Dementias 19(2), 111–116 (2004)

    Article  Google Scholar 

  16. Modares, H., Ranatunga, I., Lewis, F.L., Popa, D.O.: Optimized assistive human-robot interaction using reinforcement learning. IEEE Trans. Cybern. 46, 655–667 (2015)

    Article  Google Scholar 

  17. Pietquin, O., Lopes, M.: Machine learning for interactive systems: challenges and future trends. In: WACAI (2014)

    Google Scholar 

  18. Raya, R., Rocon, E., Urendes, E., Velasco, M.A., Clemotte, A., Ceres, R.: Assistive robots for physical and cognitive rehabilitation in cerebral palsy. In: Mohammed, S., Moreno, J.C., Kong, K., Amirat, Y. (eds.) Intelligent Assistive Robots: Recent Advances in Assistive Robotics for Everyday Activities. Springer Tracts in Advanced Robotics, vol. 106, pp. 133–156. Springer, Heidelberg (2015)

    Google Scholar 

  19. Rieser, V., Lemon, O.: Reinforcement Learning for Adaptive Dialogue Systems: A Data-driven Methodology for Dialogue Management and Natural Language Generation. Theory and Applications of Natural Language Processing. Springer, Heidelberg (2011)

    Book  Google Scholar 

  20. Senft, E., Baxter, P., Kennedy, J., Belpaeme, T.: SPARC: supervised progressively autonomous robot competencies. In: Tapus, A., André, E., Martin, J.-C., Ferland, F., Ammi, M. (eds.) Social Robotics. LNCS, vol. 9388, pp. 603–612. Springer, Heidelberg (2015)

    Chapter  Google Scholar 

  21. Tapus, A.: Improving the quality of life of people with dementia through the use of socially assistive robots. In: Advanced Technologies for Enhanced Quality of Life (AT-EQUAL 2009), pp. 81–86. IEEE (2009)

    Google Scholar 

  22. Torrey, L., Taylor, M.: Teaching on a budget: agents advising agents in reinforcement learning. In: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, pp. 1053–1060. International Foundation for Autonomous Agents and Multiagent Systems (2013)

    Google Scholar 

  23. Tsiakas, K.: Facilitating safe adaptation of interactive agents using interactive reinforcement learning. In: Companion Publication of the 21st International Conference on Intelligent User Interfaces, pp. 106–109. ACM (2016)

    Google Scholar 

  24. Tsiakas, K., Huber, M., Makedon, F.: A multimodal adaptive session manager for physical rehabilitation exercising. In: Proceedings of the 8th ACM International Conference on Pervasive Technologies Related to Assistive Environments. ACM (2015)

    Google Scholar 

  25. Wada, K., et al.: Robot therapy for elders affected by dementia. IEEE Eng. Med. Biol. Mag. 4(27), 53–60 (2008)

    Google Scholar 

Download references

Acknowledgments

This material is based upon work supported by NSF under award numbers CNS 1338118, 1035913 and by the educational program of NCSR Demokritos in collaboration with the University of Texas at Arlington.

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, University of Texas at Arlington, Arlington, USA

    Konstantinos Tsiakas & Fillia Makedon

  2. National Center for Scientific Research “Demokritos”, Athens, Greece

    Konstantinos Tsiakas, Maria Dagioglou & Vangelis Karkaletsis

Authors
  1. Konstantinos Tsiakas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maria Dagioglou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vangelis Karkaletsis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fillia Makedon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Konstantinos Tsiakas .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, The University of Kansas, Lawrence, Indiana, USA

    Arvin Agah

  2. Department of Mechanical Engineering, Qatar University, Doha, Qatar

    John-John Cabibihan

  3. School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, USA

    Ayanna M. Howard

  4. Department of Systems Engineering and Automation, University Carlos III de Madrid, madrid, Spain

    Miguel A. Salichs

  5. Department of Mechanical, Aerospace and Biomedical Engineering, University of Tennessee, Knoxville, Tennessee, USA

    Hongsheng He

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Tsiakas, K., Dagioglou, M., Karkaletsis, V., Makedon, F. (2016). Adaptive Robot Assisted Therapy Using Interactive Reinforcement Learning. In: Agah, A., Cabibihan, JJ., Howard, A., Salichs, M., He, H. (eds) Social Robotics. ICSR 2016. Lecture Notes in Computer Science(), vol 9979. Springer, Cham. https://doi.org/10.1007/978-3-319-47437-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-47437-3_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47436-6

  • Online ISBN: 978-3-319-47437-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation