Skip to main content
SpringerLink
Log in

Transparent Interactive Reinforcement Learning Using Emotional Behaviours

  • Conference paper
  • First Online:
Social Robotics (ICSR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13817))

Included in the following conference series:

Abstract

This work presents a model for improving transparency during robot learning tasks in Human-Robot Interaction scenarios. Our model puts the human in the learning loop by using two categories of robot’s emotional/behavioural reactions, one associated with the learning process of the robot and another elicited as a response to the feedback provided by the user. Preliminary results from a between-subjects study show that people empathized more with a robot expressing its emotions in both the above categories. We noticed a slight increase in the transparency of the robot while it expressed emotions during the learning process and as a response to the user. These findings highlight the importance of emotional behaviours for improving the transparency in the learning systems, which are fundamental for social learning scenarios in future humanoid robotic applications.

This work has been supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 955778, and CHIST-ERA IV COHERENT project “COllaborative HiErarchical Robotic ExplaNaTions”, and Italian PON R &I 2014-2020 - REACT-EU (CUP E65F21002920003).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arghode, V., Yalvac, B., Liew, J.: Teacher empathy and science education: a collective case study. Eurasia J. Math. Sci. Technol. Educ. 9(2), 89–99 (2013)

    Google Scholar 

  2. Baron-Cohen, S.: The essential difference: the male and female brain. In: Phi Kappa Phi Forum, vol. 85, pp. 23–26 (2005)

    Google Scholar 

  3. Breazeal, C.: Designing sociable machines. In: Socially Intelligent Agents, pp. 149–156. Springer, Heidelberg (2002). https://doi.org/10.1007/0-306-47373-9_18

  4. Broekens, J., Chetouani, M.: Towards transparent robot learning through tdrl-based emotional expressions. IEEE Trans. Affect. Comput. 12(2), 352–362 (2019)

    Article  Google Scholar 

  5. Chvátal, V.: Mastermind. Combinatorica 3(3), 325–329 (1983)

    Article  MATH  Google Scholar 

  6. Davis, M.H.: Measuring individual differences in empathy: evidence for a multidimensional approach. J. Pers. Soc. Psychol. 44(1), 113 (1983)

    Article  Google Scholar 

  7. Hindemith, L., Bruns, O., Noller, A.M., Hemion, N., Schneider, S., Vollmer, A.L.: Interactive robot task learning: Human teaching proficiency with different feedback approaches. IEEE Trans. Cogn. Dev. Syst. (2022)

    Google Scholar 

  8. Kwak, S.S., Kim, Y., Kim, E., Shin, C., Cho, K.: What makes people empathize with an emotional robot?: the impact of agency and physical embodiment on human empathy for a robot. In: 2013 IEEE RO-MAN, pp. 180–185. IEEE (2013)

    Google Scholar 

  9. Lin, J., Zhang, Q., Gomez, R., Nakamura, K., He, B., Li, G.: Human social feedback for efficient interactive reinforcement agent learning. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 706–712. IEEE (2020)

    Google Scholar 

  10. Löffler, D., Schmidt, N., Tscharn, R.: Multimodal expression of artificial emotion in social robots using color, motion and sound. In: 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 334–343. IEEE (2018)

    Google Scholar 

  11. Matarese, M., Sciutti, A., Rea, F., Rossi, S.: Toward robots’ behavioral transparency of temporal difference reinforcement learning with a human teacher. IEEE Trans. Hum.-Mach. Syst. 51(6), 578–589 (2021)

    Article  Google Scholar 

  12. McAllister, G., Irvine, J.J.: The role of empathy in teaching culturally diverse students: a qualitative study of teachers’ beliefs. J. Teach. Educ. 53(5), 433–443 (2002)

    Article  Google Scholar 

  13. Pekrun, R.: Emotions and learning. Educ. Pract. Ser. 24(1), 1–31 (2014)

    Google Scholar 

  14. Rossi, A., Dautenhahn, K., Lee Koay, K., Walters, M.L.: How social robots influence people’s trust in critical situations. In: RO-MAN 2020, pp. 1020–1025 (2020)

    Google Scholar 

  15. Rossi, A., Scheunemann, M.M., L’Arco, G., Rossi, S.: Evaluation of a humanoid robot’s emotional gestures for transparent interaction. In: Li, H., et al. (eds.) ICSR 2021. LNCS (LNAI), vol. 13086, pp. 397–407. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90525-5_34

    Chapter  Google Scholar 

  16. Suay, H.B., Chernova, S.: Effect of human guidance and state space size on interactive reinforcement learning. In: 2011 Ro-Man, pp. 1–6. IEEE (2011)

    Google Scholar 

  17. Thomaz, A.L., Hoffman, G., Breazeal, C.: Real-time interactive reinforcement learning for robots. In: AAAI 2005 Workshop on Human Comprehensible Machine Learning, pp. 9–13 (2005)

    Google Scholar 

  18. Tiedens, L.Z., Linton, S.: Judgment under emotional certainty and uncertainty: the effects of specific emotions on information processing. J. Pers. Social Psychol. 81(6), 973 (2001)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Interdepartmental Center for Advances in Robotic Surgery - ICAROS, University of Naples Federico II, Naples, Italy

    Georgios Angelopoulos, Alessandra Rossi & Silvia Rossi

  2. Department of Electrical Engineering and Information Technologies - DIETI, University of Naples Federico II, Naples, Italy

    Alessandra Rossi, Gianluca L’Arco & Silvia Rossi

Authors
  1. Georgios Angelopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandra Rossi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gianluca L’Arco
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Silvia Rossi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Georgios Angelopoulos .

Editor information

Editors and Affiliations

  1. University of Florence, Florence, Italy

    Filippo Cavallo

  2. Qatar University, Doha, Qatar

    John-John Cabibihan

  3. University of Florence, Florence, Italy

    Laura Fiorini

  4. University of Florence, Florence, Italy

    Alessandra Sorrentino

  5. Wichita State University, Wichita, KS, USA

    Hongsheng He

  6. Qingdao University, Qingdao, China

    Xiaorui Liu

  7. National Institute of Advanced Industrial Science and Technology, Tsukuba, Japan

    Yoshio Matsumoto

  8. National University of Singapore, Singapore, Singapore

    Shuzhi Sam Ge

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Angelopoulos, G., Rossi, A., L’Arco, G., Rossi, S. (2022). Transparent Interactive Reinforcement Learning Using Emotional Behaviours. In: Cavallo, F., et al. Social Robotics. ICSR 2022. Lecture Notes in Computer Science(), vol 13817. Springer, Cham. https://doi.org/10.1007/978-3-031-24667-8_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-24667-8_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-24666-1

  • Online ISBN: 978-3-031-24667-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation