skip to main content
10.1145/3543507.3583438acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

EmpMFF: A Multi-factor Sequence Fusion Framework for Empathetic Response Generation

Published: 30 April 2023 Publication History

Abstract

Empathy is one of the fundamental abilities of dialog systems. In order to build more intelligent dialogue systems, it’s important to learn how to demonstrate empathy toward others. Existing studies focus on identifying and leveraging the user’s coarse emotion to generate empathetic responses. However, human emotion and dialog act (e.g., intent) evolve as the talk goes along in an empathetic dialogue. This leads to the generated responses with very different intents from the human responses. As a result, empathy failure is ultimately caused. Therefore, using fine-grained emotion and intent sequential data on conversational emotions and dialog act is crucial for empathetic response generation. On the other hand, existing empathy models overvalue the empathy of responses while ignoring contextual relevance, which results in repetitive model-generated responses. To address these issues, we propose a Multi-Factor sequence Fusion framework (EmpMFF) based on conditional variational autoencoder. To generate empathetic responses, the proposed EmpMFF encodes a combination of contextual, emotion, and intent information into a continuous latent variable, which is then fed into the decoder. Experiments on the EmpatheticDialogues benchmark dataset demonstrate that EmpMFF exhibits exceptional performance in both automatic and human evaluations.

References

[1]
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).
[2]
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: Commonsense Transformers for Automatic Knowledge Graph Construction. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 4762–4779. https://doi.org/10.18653/v1/p19-1470
[3]
Kris Cao and Stephen Clark. 2017. Latent Variable Dialogue Models and their Diversity. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, 2017, Volume 2: Short Papers, Mirella Lapata, Phil Blunsom, and Alexander Koller (Eds.). Association for Computational Linguistics, 182–187. https://doi.org/10.18653/v1/e17-2029
[4]
Mao Yan Chen, Siheng Li, and Yujiu Yang. 2022. EmpHi: Generating Empathetic Responses with Human-like Intents. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, Marine Carpuat, Marie-Catherine de Marneffe, and Iván Vladimir Meza Ruíz (Eds.). Association for Computational Linguistics, 1063–1074. https://doi.org/10.18653/v1/2022.naacl-main.78
[5]
Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan S. Cowen, Gaurav Nemade, and Sujith Ravi. 2020. GoEmotions: A Dataset of Fine-Grained Emotions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 4040–4054. https://doi.org/10.18653/v1/2020.acl-main.372
[6]
Le Fang, Tao Zeng, Chaochun Liu, Liefeng Bo, Wen Dong, and Changyou Chen. 2021. Transformer-based Conditional Variational Autoencoder for Controllable Story Generation. CoRR abs/2101.00828 (2021). arXiv:2101.00828https://arxiv.org/abs/2101.00828
[7]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144.
[8]
Sevgi Coşkun Keskin. 2014. From what isn’t empathy to empathic learning process. Procedia-Social and Behavioral Sciences 116 (2014), 4932–4938.
[9]
Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1312.6114
[10]
Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, and Jianfeng Gao. 2020. Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 4678–4699. https://doi.org/10.18653/v1/2020.emnlp-main.378
[11]
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A Diversity-Promoting Objective Function for Neural Conversation Models. In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, Kevin Knight, Ani Nenkova, and Owen Rambow (Eds.). The Association for Computational Linguistics, 110–119. https://doi.org/10.18653/v1/n16-1014
[12]
Qintong Li, Hongshen Chen, Zhaochun Ren, Pengjie Ren, Zhaopeng Tu, and Zhumin Chen. 2020. EmpDG: Multi-resolution Interactive Empathetic Dialogue Generation. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, Donia Scott, Núria Bel, and Chengqing Zong (Eds.). International Committee on Computational Linguistics, 4454–4466. https://doi.org/10.18653/v1/2020.coling-main.394
[13]
Qintong Li, Piji Li, Zhaochun Ren, Pengjie Ren, and Zhumin Chen. 2022. Knowledge Bridging for Empathetic Dialogue Generation. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022. AAAI Press, 10993–11001. https://ojs.aaai.org/index.php/AAAI/article/view/21347
[14]
Zhaojiang Lin, Andrea Madotto, Jamin Shin, Peng Xu, and Pascale Fung. 2019. MoEL: Mixture of Empathetic Listeners. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 121–132. https://doi.org/10.18653/v1/D19-1012
[15]
Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum¿id=Bkg6RiCqY7
[16]
Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander F. Gelbukh, Rada Mihalcea, and Soujanya Poria. 2020. MIME: MIMicking Emotions for Empathetic Response Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 8968–8979. https://doi.org/10.18653/v1/2020.emnlp-main.721
[17]
Ana Paiva, Iolanda Leite, Hana Boukricha, and Ipke Wachsmuth. 2017. Empathy in Virtual Agents and Robots: A Survey. ACM Trans. Interact. Intell. Syst. 7, 3 (2017), 11:1–11:40. https://doi.org/10.1145/2912150
[18]
Martin Popel and Ondrej Bojar. 2018. Training Tips for the Transformer Model. Prague Bull. Math. Linguistics 110 (2018), 43–70. http://ufal.mff.cuni.cz/pbml/110/art-popel-bojar.pdf
[19]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
[20]
Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 5370–5381. https://doi.org/10.18653/v1/p19-1534
[21]
Sahand Sabour, Chujie Zheng, and Minlie Huang. 2022. CEM: Commonsense-Aware Empathetic Response Generation. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022. AAAI Press, 11229–11237. https://ojs.aaai.org/index.php/AAAI/article/view/21373
[22]
Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. 2017. A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, Satinder Singh and Shaul Markovitch (Eds.). AAAI Press, 3295–3301. http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14567
[23]
Liuping Wang, Dakuo Wang, Feng Tian, Zhenhui Peng, Xiangmin Fan, Zhan Zhang, Mo Yu, Xiaojuan Ma, and Hongan Wang. 2021. CASS: Towards Building a Social-Support Chatbot for Online Health Community. Proc. ACM Hum. Comput. Interact. 5, CSCW1 (2021), 1–31. https://doi.org/10.1145/3449083
[24]
Anuradha Welivita and Pearl Pu. 2020. A Taxonomy of Empathetic Response Intents in Human Social Conversations. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, Donia Scott, Núria Bel, and Chengqing Zong (Eds.). International Committee on Computational Linguistics, 4886–4899. https://doi.org/10.18653/v1/2020.coling-main.429
[25]
Rohola Zandie and Mohammad H. Mahoor. 2020. EmpTransfo: A Multi-Head Transformer Architecture for Creating Empathetic Dialog Systems. In Proceedings of the Thirty-Third International Florida Artificial Intelligence Research Society Conference, Originally to be held in North Miami Beach, Florida, USA, May 17-20, 2020, Roman Barták and Eric Bell (Eds.). AAAI Press, 276–281. https://aaai.org/ocs/index.php/FLAIRS/FLAIRS20/paper/view/18446
[26]
Tiancheng Zhao, Ran Zhao, and Maxine Eskénazi. 2017. Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 654–664. https://doi.org/10.18653/v1/P17-1061
[27]
Chujie Zheng, Yong Liu, Wei Chen, Yongcai Leng, and Minlie Huang. 2021. CoMAE: A Multi-factor Hierarchical Framework for Empathetic Response Generation. In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021(Findings of ACL, Vol. ACL/IJCNLP 2021), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 813–824. https://doi.org/10.18653/v1/2021.findings-acl.72

Cited By

View all
  • (2024)Cross-Task Multimodal Reinforcement for Long Tail Next POI RecommendationIEEE Transactions on Multimedia10.1109/TMM.2023.329072326(1996-2005)Online publication date: 2024
  • (2024)TriKF: Triple-Perspective Knowledge Fusion Network for Empathetic Question GenerationIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.341882011:6(7186-7199)Online publication date: Dec-2024
  • (2024)A Map of Exploring Human Interaction Patterns with LLM: Insights into Collaboration and CreativityArtificial Intelligence in HCI10.1007/978-3-031-60615-1_5(60-85)Online publication date: 29-Jun-2024
  • Show More Cited By

Index Terms

  1. EmpMFF: A Multi-factor Sequence Fusion Framework for Empathetic Response Generation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WWW '23: Proceedings of the ACM Web Conference 2023
    April 2023
    4293 pages
    ISBN:9781450394161
    DOI:10.1145/3543507
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 30 April 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Dialong System
    2. Empathetic Response
    3. Pre-trained Language Model

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • NSFC

    Conference

    WWW '23
    Sponsor:
    WWW '23: The ACM Web Conference 2023
    April 30 - May 4, 2023
    TX, Austin, USA

    Acceptance Rates

    Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)82
    • Downloads (Last 6 weeks)7
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Cross-Task Multimodal Reinforcement for Long Tail Next POI RecommendationIEEE Transactions on Multimedia10.1109/TMM.2023.329072326(1996-2005)Online publication date: 2024
    • (2024)TriKF: Triple-Perspective Knowledge Fusion Network for Empathetic Question GenerationIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.341882011:6(7186-7199)Online publication date: Dec-2024
    • (2024)A Map of Exploring Human Interaction Patterns with LLM: Insights into Collaboration and CreativityArtificial Intelligence in HCI10.1007/978-3-031-60615-1_5(60-85)Online publication date: 29-Jun-2024
    • (2023)Personalized Re-ranking for Recommendation with Mask PretrainingData Science and Engineering10.1007/s41019-023-00219-68:4(357-367)Online publication date: 2-Sep-2023
    • (2023)UaMC: user-augmented conversation recommendation via multi-modal graph learning and context miningWorld Wide Web10.1007/s11280-023-01219-226:6(4109-4129)Online publication date: 19-Dec-2023
    • (2023)Continuous frequent contact detection over moving objectsGeoinformatica10.1007/s10707-023-00501-928:2(271-290)Online publication date: 17-Jul-2023

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media