Skip to main content
SpringerLink
Log in

SINN: A speaker influence aware neural network model for emotion detection in conversations

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Inferring the sentiment polarity or emotion category of subjective text is the fundamental task of sentiment analysis. Recently, emotion detection in conversations that considering context utterances has emerged as a very important and challenging task in this line of research. Most existing studies do not distinguish different speakers in a dialog and fail to characterize inter-speaker dependencies for emotion detection. In this paper, we propose a S peaker I nfluence aware N eural N etwork model (dubbed as SINN) to predict the emotion of the last utterance in a conversation, which explicitly models the self and inter-speaker influences of historical utterances with GRUs (Gated Recurrent Units) and hierarchical attention matching network. Moreover, the empathy phenomenon is also considered by an emotion state tracking component in SINN. Finally, the target utterance representation is enhanced by speaker influence aware context modeling, where an attention mechanism is used to extract the most relevant features for emotion classification. We construct a large-scale multi-turn Chinese dialog dataset WBEmoDialog, where each utterance is manually annotated with an emotion label. Extensive experiments are conducted on public available DailyDialog dataset as well as our constructed WBEmoDialog dataset, and the results show that our model can achieve better or comparable performance with the strong baseline methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://web.wechat.com/

  2. https://www.twitter.com/

  3. https://www.weibo.com/

  4. WBEmoDialog dataset is released at https://github.com/YangXiaocui1215/WBEmoDialog

  5. http://conference.cipsc.org.cn/smp2019/evaluation.html

  6. https://github.com/ZaneMuir/DLUT-Emotionontology

  7. https://github.com/fxsjy/jieba

References

  1. Akbari, H., Sadiq, M.T., Rehman, A.U.: Classification of normal and depressed EEG signals based on centered correntropy of rhythms in empirical wavelet transform domain. Health Inf. Sci. Syst. 9(1), 9 (2021)

    Article  Google Scholar 

  2. Becker, K., Moreira, V.P., dos Santos, A.G.L.: Multilingual emotion classification using supervised learning: Comparative experiments. Inf. Process. Manag. 53(3), 684–704 (2017)

    Article  Google Scholar 

  3. Busso, C., Bulut, M., Lee, C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J.N., Lee, S., Narayanan, S.S.: IEMOCAP: interactive emotional dyadic motion capture database. Lang. Resour. Eval. 42(4), 335–359 (2008)

    Article  Google Scholar 

  4. Chatterjee, A., Gupta, U., Chinnakotla, M.K., Srikanth, R., Galley, M., Agrawal, P.: Understanding emotions in text using deep learning and big data. Comput. Hum. Behav. 93, 309–317 (2019)

    Article  Google Scholar 

  5. Cho, K., van Merrienboer, B., Gu̇lċehre, Ç., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp 1724–1734 (2014)

  6. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv:1406.1078 (2014)

  7. Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3-4), 169–200 (1992)

    Article  Google Scholar 

  8. Feng, J., Rao, Y., Xie, H., Wang, F.L., Li, Q.: User group based emotion detection and topic discovery over short text. World Wide Web 23(3), 1553–1587 (2020)

    Article  Google Scholar 

  9. Feng, S., Wang, Y., Liu, L., Wang, D., Yu, G.: Attention based hierarchical LSTM network for context-aware microblog sentiment classification. World Wide Web 22(1), 59–81 (2019)

    Article  Google Scholar 

  10. Feng, S., Wang, Y., Song, K., Wang, D., Yu, G.: Detecting multiple coexisting emotions in microblogs with convolutional neural networks. Cogn. Comput. 10(1), 136–155 (2018)

    Article  Google Scholar 

  11. Ferraro, G., Gee, B.L., Ji, S., Salvador-Carulla, L.: Lightme: analysing language in internet support groups for mental health. Health Inf. Sci. Syst. 8(1), 34 (2020)

    Article  Google Scholar 

  12. Fung, P., Bertero, D., Wan, Y., Dey, A., Chan, R.H.Y., Siddique, F.B., Yang, Y., Wu, C., Lin, R.: Towards empathetic human-robot interactions. In: Computational Linguistics and Intelligent Text Processing - 17th International Conference, CICLing 2016, Konya, Turkey, April 3-9, 2016, Revised Selected Papers, Part II, pp 173–193 (2016)

  13. Gui, L., Lin, H., Lin, Y., Liu, S.: Detection and extraction of hot topics on chinese microblogs. Cogn. Comput. 8(4), 577–586 (2016)

    Article  Google Scholar 

  14. Gupta, U., Chatterjee, A., Srikanth, R., Agrawal, P.: A sentiment-and-semantics-based approach for emotion detection in textual conversations. arXiv:1707.06996 (2017)

  15. Hazarika, D., Poria, S., Mihalcea, R., Cambria, E., Zimmermann, R.: Icon: Interactive conversational memory network for multimodal emotion detection. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 2594–2604 (2018)

  16. Hazarika, D., Poria, S., Zadeh, A., Cambria, E., Morency, L.P., Zimmermann, R.: Conversational memory network for emotion recognition in dyadic dialogue videos. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp 2122–2132 (2018)

  17. Hill, C.E., O’Brien, K.M.: Helping skills: Facilitating exploration, insight, and action. American Psychological Association, Washington (1999)

    Google Scholar 

  18. Hossain, M.D., Kabir, M.A., Anwar, A., Islam, M.Z.: Detecting autism spectrum disorder using machine learning techniques. Health Inf. Sci. Syst. 9(1), 17 (2021)

    Article  Google Scholar 

  19. Hsu, C., Chen, S., Kuo, C., Huang, T.K., Ku, L.: Emotionlines: An emotion corpus of multi-party conversations. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, May 7-12, 2018 (2018)

  20. Huang, M., Cao, Y., Dong, C. arXiv:1605.01478 (2016)

  21. Husin, N., Abdullah, M.T., Mahmod, R.: A systematic literature review for topic detection in chat conversation for cyber-crime investigation. Int. J. Digit. Content Technol. Appl. 8(3), 22 (2014)

    Google Scholar 

  22. Inui, K., Jiang, J., Ng, V., Wan, X. (eds.): Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019 Association for Computational Linguistics (2019)

  23. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp 1746–1751 (2014)

  24. Kuppens, P., Allen, N.B., Sheeber, L.B.: Emotional inertia and psychological maladjustment. Psychol. Sci. 21(7), 984–991 (2010)

    Article  Google Scholar 

  25. Li, Y., Su, H., Shen, X., Li, W., Cao, Z., Niu, S.: Dailydialog: A manually labelled multi-turn dialogue dataset. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, Taipei, Taiwan, November 27 - December 1, 2017 - Volume 1: Long Papers, pp 986–995 (2017)

  26. Liu, S., Lee, I.: Extracting features with medical sentiment lexicon and position encoding for drug reviews. Health Inf. Sci. Syst. 7(1), 11 (2019)

    Article  Google Scholar 

  27. Liu, S., Zheng, C., Demasi, O., Sabour, S., Li, Y., Yu, Z., Jiang, Y., Huang, M.: Towards emotional support dialog systems. arXiv:2106.01144 (2021)

  28. Luo, L., Yang, H., Chin, F.Y.: Emotionx-dlc: Self-attentive bilstm for detecting sequential emotions in dialogue. arXiv:1806.07039 (2018)

  29. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings (2013)

  30. Morency, L., Bohus, D., Aghajan, H.K., Cassell, J., Nijholt, A., Epps, J. (eds.): International Conference on Multimodal Interaction, ICMI ’12, Santa Monica, CA, USA, October 22-26, 2012. ACM (2012)

  31. Morris, M.W., Keltner, D.: How emotions work: The social functions of emotional expression in negotiations. Res. Organ. Behav. 22, 1–50 (2000)

    Google Scholar 

  32. Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. arXiv:1802.05365 (2018)

  33. Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., Mihalcea, R.: MELD: A multimodal multi-party dataset for emotion recognition in conversations. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp 527–536 (2019)

  34. Purpura, A., Masiero, C., Silvello, G., Susto, G.A.: Supervised lexicon extraction for emotion classification. In: Companion of The 2019 World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019, pp 1071–1078 (2019)

  35. Rao, Y., Lei, J., Wenyin, L., Li, Q., Chen, M.: Building emotional dictionary for sentiment analysis of online news. World Wide Web 17(4), 723–742 (2014)

    Article  Google Scholar 

  36. Rashkin, H., Smith, E.M., Li, M., Boureau, Y.: Towards empathetic open-domain conversation models: A new benchmark and dataset. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp 5370–5381 (2019)

  37. Ren, Y., Zhang, Y., Zhang, M., Ji, D.: Context-sensitive twitter sentiment classification using neural network. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)

  38. Shen, C., Sun, C., Wang, J., Kang, Y., Li, S., Liu, X., Si, L., Zhang, M., Zhou, G.: Sentiment classification towards question-answering with hierarchical matching network. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 3654–3663 (2018)

  39. Shen, L., Feng, Y.: CDL: curriculum dual learning for emotion-controllable response generation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp 556–566 (2020)

  40. Song, K., Bing, L., Gao, W., Lin, J., Zhao, L., Wang, J., Sun, C., Liu, X., Zhang, Q.: Using customer service dialogues for satisfaction analysis with context-assisted multiple instance learning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pp 198–207 (2019)

  41. Song, K., Feng, S., Gao, W., Wang, D., Chen, L., Zhang, C.: Build emotion lexicon from microblogs by combining effects of seed words and emoticons in a heterogeneous graph. In: Proceedings of the 26th ACM Conference on Hypertext & Social Media, HT 2015, Guzelyurt, TRNC, Cyprus, September 1-4, 2015, pp 283–292 (2015)

  42. Sun, H., Lin, Z., Zheng, C., Liu, S., Huang, M.: Psyqa: A chinese dataset for generating long counseling text for mental health support. arXiv:2106.01702 (2021)

  43. Tago, K., Takagi, K., Kasuya, S., Jin, Q.: Analyzing influence of emotional tweets on user relationships using naive bayes and dependency parsing. World Wide Web 22(3), 1263–1278 (2019)

    Article  Google Scholar 

  44. Thabtah, F.A., Abdelhamid, N., Peebles, D.: A machine learning autism classification based on logistic regression analysis. Health Inf. Sci. Syst. 7 (1), 12 (2019)

    Article  Google Scholar 

  45. Tokhisa, R., Inui, K., Matsumoto, Y.: Emotion classification using massive examples extracted from the web. In: COLING 2008, 22nd International Conference on Computational Linguistics, Proceedings of the Conference, 18-22 August 2008, Manchester, UK, pp 881–888 (2008)

  46. Vanzo, A., Croce, D., Basili, R.: A context-based model for sentiment analysis in twitter. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp 2345–2354 (2014)

  47. Wang, Y., Feng, S., Wang, D., Zhang, Y., Yu, G.: Context-aware chinese microblog sentiment classification with bidirectional lstm. In: Asia-Pacific Web Conference, pp 594–606. Springer (2016)

  48. Wei, J., Feng, S., Wang, D., Zhang, Y., Li, X.: Attentional neural network for emotion detection in conversations with speaker influence awareness. In: Tang, J., Kan, M., Zhao, D., Li, S., Zan, H. (eds.) Natural Language Processing and Chinese Computing - 8th CCF International Conference, NLPCC 2019, Dunhuang, China, October 9-14, 2019, Proceedings, Part II, Lecture Notes in Computer Science, vol. 11839, pp 287–297. Springer (2019)

  49. Wen, S., Wan, X.: Emotion classification in microblog texts using class sequential rules. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pp 187–193 (2014)

  50. Yang, Y., Zhou, D., He, Y., Zhang, M.: Interpretable relevant emotion ranking with event-driven attention. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pp 177–187 (2019)

  51. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 1480–1489 (2016)

  52. Zhang, L., Chen, C.: Sentiment classification with convolutional neural networks: An experimental study on a large-scale chinese conversation corpus. In: 2016 12th International Conference on Computational Intelligence and Security (CIS), pp 165–169. IEEE (2016)

  53. Zhang, Y., Fu, J., She, D., Zhang, Y., Wang, S., Yang, J.: Text emotion distribution learning via multi-task convolutional neural network. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pp 4595–4601 (2018)

  54. Zhou, H., Huang, M., Zhang, T., Zhu, X., Liu, B.: Emotional chatting machine: Emotional conversation generation with internal and external memory. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pp 730–739 (2018)

  55. Zhou, X., Wang, W.Y.: Mojitalk: Generating emotional responses at scale. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, pp 1128–1137 (2018)

  56. Zhou, Y., Li, C., Xu, B., Xu, J., Yang, L., Xu, B.: Constructing a Chinese conversation corpus for sentiment analysis. In: Natural Language Processing and Chinese Computing - 6th CCF International Conference, NLPCC 2017, Dalian, China, November 8-12, 2017, Proceedings, pp 579–590 (2017)

Download references

Acknowledgements

The work was supported by the National Key R&D Program of China under Grant 2018YFB1004700, National Natural Science Foundation of China (61872074, 61772122), and the Fundamental Research Funds for the Central Universities (N180716010). This paper is a substantial extension of our previous work in [48]. In this paper, a large-scale Chinese dialog dataset WBEmoDialog is constructed, and the proposed SINN model is evaluated on the new dataset. More baselines, ablation experiments and discussions are included in the extended version. We thank reviewers for their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Northeastern University, No.195 Chuangxin Road, Hunnan District, Shenyang, China

    Shi Feng, Jia Wei, Daling Wang, Xiaocui Yang, Zhenfei Yang, Yifei Zhang & Ge Yu

Authors
  1. Shi Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daling Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaocui Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhenfei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yifei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ge Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shi Feng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, S., Wei, J., Wang, D. et al. SINN: A speaker influence aware neural network model for emotion detection in conversations. World Wide Web 24, 2019–2048 (2021). https://doi.org/10.1007/s11280-021-00954-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-021-00954-8

Keywords

Search

Navigation