skip to main content
10.1145/3472306.3478344acmconferencesArticle/Chapter ViewAbstractPublication PagesivaConference Proceedingsconference-collections
research-article

Enhancing Conversational Agents with Empathic Abilities

Published: 14 September 2021 Publication History

Abstract

Conversational agents are getting increasingly popular and find applications in health and customer services. Conversations in these fields are often emotionally charged. It is, therefore, necessary to handle the conversation with some degree of empathy to be effective. In this work, we leverage advances in the field of natural language processing to create a dialogue system that can convincingly generate empathic responses to text-based messages. To improve the system's ability to converse with empathy, we train the language model on empathic conversations and inject additional emotional information in the response generation. We propose two chatbots: a benchmark bot and an empathic bot. Additionally, we implement an emotion classifier that allows us to predict the emotional state of text-based messages. We evaluate both chatbots in quantitative studies and compare them with human responses in qualitative studies involving human judges. Our evaluation shows that our empathic chatbot outperforms the benchmark bot and even the human-generated responses in terms of perceived empathy. Additionally, we achieve state-of-the-art results in terms of response quality using transformer-based language models. Finally we report that we can double the initial performance of the emotion classifier using undersampling techniques, yielding a final F1-score of 0.81 in six basic emotions.

References

[1]
Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. 2020. Towards a Human-like Open-Domain Chatbot. arXiv:2001.09977 [cs.CL]
[2]
Rafael E. Banchs. 2017. On the construction of more human-like chatbots: Affect and emotion analysis of movie dialogue data. In 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 1364--1367. https://doi.org/10.1109/apsipa.2017.8282245
[3]
Aaron Ben-Zeev. 2000. The Complexity of Emotions. In The Subtlety of Emotions. The MIT Press. https://doi.org/10.7551/mitpress/6548.003.0004
[4]
C. Daryl Cameron. 2018. Motivating empathy: Three methodological recommendations for mapping empathy. Social and Personality Psychology Compass 12, 11 (oct 2018), e12418. https://doi.org/10.1111/spc3.12418
[5]
Jacky Casas, Marc-Olivier Tricot, Omar Abou Khaled, Elena Mugellini, and Philippe Cudré-Mauroux. 2020. Trends & Methods in Chatbot Evaluation. In Companion Publication of the 2020 International Conference on Multimodal Interaction (Virtual Event, Netherlands). Association for Computing Machinery, New York, NY, USA, 280--286. https://doi.org/10.1145/3395035.3425319
[6]
Yin Hei Chan and Andrew Kwok Fai Lui. 2018. Encoding emotional information for sequence-to-sequence response generation. In 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD). IEEE, 113--116. https://doi.org/10.1109/icaibd.2018.8396177
[7]
Benjamin M.P. Cuff, Sarah J. Brown, Laura Taylor, and Douglas J. Howat. 2016. Empathy: A Review of the Concept. Emotion Review 8, 2 (2016), 144--153. https://doi.org/10.1177/1754073914558466
[8]
Paul Ekman. 1992. An argument for basic emotions. Cognition and Emotion 6, 3--4 (may 1992), 169--200. https://doi.org/10.1080/02699939208411068
[9]
Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. ACL, 1615--1625. https://doi.org/10.18653/v1/d17-1169
[10]
Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019. The Curious Case of Neural Text Degeneration. CoRR abs/1904.09751 (2019). arXiv:1904.09751 http://arxiv.org/abs/1904.09751
[11]
Qintong Li, Hongshen Chen, Zhaochun Ren, Pengjie Ren, Zhaopeng Tu, and Zhumin Chen. 2020. EmpDG: Multi-resolution Interactive Empathetic Dialogue Generation. In Proceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics, Barcelona, Spain (Online), 4454--4466. https://doi.org/10.18653/v1/2020.coling-main.394
[12]
Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Asian Federation of Natural Language Processing, Taipei, Taiwan, 986--995. https://aclanthology.org/I17-1099
[13]
Zhaojiang Lin, Peng Xu, Genta Indra Winata, Farhad Bin Siddique, Zihan Liu, Jamin Shin, and Pascale Fung. 2020. CAiRE: An End-to-End Empathetic Chatbot. Proceedings of the AAAI Conference on Artificial Intelligence 34, 09 (Apr. 2020), 13622--13623. https://doi.org/10.1609/aaai.v34i09.7098
[14]
Bingjie Liu and S. Shyam Sundar. 2018. Should Machines Express Sympathy and Empathy? Experiments with a Health Advice Chatbot. Cyberpsychology, Behavior, and Social Networking 21, 10 (oct 2018), 625--636. https://doi.org/10.1089/cyber.2018.0110
[15]
Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, and Soujanya Poria. 2020. MIME: MIMicking Emotions for Empathetic Response Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 8968--8979. https://doi.org/10.18653/v1/2020.emnlp-main.721
[16]
Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019. MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 527--536. https://doi.org/10.18653/v1/P19-1050
[17]
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training.
[18]
Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 5370--5381. https://doi.org/10.18653/v1/P19-1534
[19]
Armin Seyeditabari, Narges Tabari, and Wlodek Zadrozny. 2018. Emotion Detection in Text: a Review. CoRR abs/1806.00674 (2018). arXiv:1806.00674 http://arxiv.org/abs/1806.00674
[20]
Néna Roa Seïler and Paul Craig. 2016. Empathetic Technology. In Emotions, Technology, and Design. Elsevier, 55--81. https://doi.org/10.1016/b978-0-12-801872-9.00004-1
[21]
Weiyan Shi and Zhou Yu. 2018. Sentiment Adaptive End-to-End Dialog Systems. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Melbourne, Australia, 1509--1519. https://doi.org/10.18653/v1/P18-1140
[22]
Timo Spring, Jacky Casas, Karl Daher, Elena Mugellini, and Omar Abou Khaled. 2019. Empathic response generation in chatbots. In Proceedings of 4th Swiss Text Analytics Conference (SwissText 2019), 18--19 June 2019, Winterthur, Switzerland. 18-19 June 2019, CEUR-WS. http://ceur-ws.org/Vol-2458/paper1.pdf
[23]
Xiao Sun, Xiaoqi Peng, and Shuai Ding. 2017. Emotional Human-Machine Conversation Generation Based on Long Short-Term Memory. Cognitive Computation 10, 3 (dec 2017), 389--397. https://doi.org/10.1007/s12559-017-9539-4
[24]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. CoRR (2017). arXiv:1706.03762 http://arxiv.org/abs/1706.03762
[25]
Dennis L Wilson. 1972. Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man, and Cybernetics 2, 3 (1972), 408--421.
[26]
Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019. TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents. CoRR abs/1901.08149 (2019). arXiv:1901.08149 http://arxiv.org/abs/1901.08149
[27]
Anbang Xu, Zhe Liu, Yufan Guo, Vibha Sinha, and Rama Akkiraju. 2017. A New Chatbot for Customer Service on Social Media. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. ACM Press. https://doi.org/10.1145/3025453.3025496
[28]
Ozge Nilay Yalcin. 2019. Evaluating Empathy in Artificial Agents. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE. https://doi.org/10.1109/acii.2019.8925498
[29]
Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2017. Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory. CoRR abs/1704.01074 (2017). arXiv:1704.01074 http://arxiv.org/abs/1704.01074
[30]
Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The Design and Implementation of XiaoIce, an Empathetic Social Chatbot. Computational Linguistics (jan 2020), 1--62. https://doi.org/10.1162/coli_a_00368
[31]
Zhiheng Zhou, Man Lan, and Yuanbin Wu. 2018. A Neural Generation-based Conversation Model Using Fine-grained Emotion-guide Attention. In 2018 International Joint Conference on Neural Networks (IJCNN). IEEE. https://doi.org/10.1109/ijcnn.2018.8488988

Cited By

View all
  • (2024)Algorithms for Empathy: Using Machine Learning to Categorize Common Empathetic Traits Across Professional and Peer-Based ConversationsCureus10.7759/cureus.57719Online publication date: 6-Apr-2024
  • (2024)Evaluating the Potential and Pitfalls of AI-Powered Conversational Agents as Humanlike Virtual Health Carers in the Remote Management of Noncommunicable Diseases: Scoping ReviewJournal of Medical Internet Research10.2196/5611426(e56114)Online publication date: 16-Jul-2024
  • (2024)Situating Empathy in HCI/CSCW: A Scoping ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36870528:CSCW2(1-37)Online publication date: 8-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IVA '21: Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents
September 2021
238 pages
ISBN:9781450386197
DOI:10.1145/3472306
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. affective computing
  2. artificial intelligence
  3. conversational agent
  4. emotions
  5. empathy
  6. human-computer interaction
  7. intelligent interface
  8. language model
  9. natural language processing
  10. virtual agent

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

IVA '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 53 of 196 submissions, 27%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)274
  • Downloads (Last 6 weeks)23
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Algorithms for Empathy: Using Machine Learning to Categorize Common Empathetic Traits Across Professional and Peer-Based ConversationsCureus10.7759/cureus.57719Online publication date: 6-Apr-2024
  • (2024)Evaluating the Potential and Pitfalls of AI-Powered Conversational Agents as Humanlike Virtual Health Carers in the Remote Management of Noncommunicable Diseases: Scoping ReviewJournal of Medical Internet Research10.2196/5611426(e56114)Online publication date: 16-Jul-2024
  • (2024)Situating Empathy in HCI/CSCW: A Scoping ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36870528:CSCW2(1-37)Online publication date: 8-Nov-2024
  • (2024)Exploring the Use of Large Language Model-Driven Chatbots in Virtual Reality to Train Autistic Individuals in Job Communication SkillsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651996(1-7)Online publication date: 11-May-2024
  • (2024)EmpathiCH: Scrutinizing Empathy-Centric Design Beyond the IndividualExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636297(1-7)Online publication date: 11-May-2024
  • (2024)Response Generation in Social Network With Topic and Emotion ConstraintsIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.339780211:5(6592-6604)Online publication date: Oct-2024
  • (2024)Reliable uncertainty estimation in emotion recognition in conversation using conformal prediction frameworkNatural Language Processing10.1017/nlp.2024.48(1-24)Online publication date: 30-Oct-2024
  • (2024)A formal understanding of computational empathy in interactive agentsCognitive Systems Research10.1016/j.cogsys.2023.10120385:COnline publication date: 2-Jul-2024
  • (2024)Beyond Factualism: A Study of LLM Calibration Through the Lens of Conversational Emotion RecognitionAI 2024: Advances in Artificial Intelligence10.1007/978-981-96-0348-0_15(198-212)Online publication date: 25-Nov-2024
  • (2023)Empathetic Response Generation Based on Plug-and-Play Mechanism With Empathy PerturbationIEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2023.327727431(2032-2042)Online publication date: 2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media