Skip to main content
SpringerLink
Log in

An emotion-based responding model for natural language conversation

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

As an important task of artificial intelligence, natural language conversation has attracted wide attention of researchers in natural language processing. Existing works in this field mainly focus on consistency of neural response generation whilst ignoring the effect of emotion state on the response generation. In this paper, we propose an Emotion-based natural language Responding Model (ERM) to address the challenging issue in conversation. ERM encodes the emotion state of the conversation as distributed embedding into the process of response generation, redefines an objective function that jointly trains our model and introduces a novel re-rank function to select the appropriate response. Experimental results on Chinese conversation dataset show that our method yields qualitative performance improvements in the Perplexity (PPL), Word Error-rate (WER) and Bilingual Evaluation Understudy (BLEU) compared with the baseline sequence-to-sequence (Seq2Seq) model, and achieves better performance than the state-of-the-art in terms of emotion and content consistency of the response.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7

Similar content being viewed by others

Notes

  1. https://raw.githubusercontent.com/BUPTLdy/Sentiment-Analysis/master/data/

  2. http://www.datatang.com/data/46475

References

  1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M.: Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467 (2016)

  2. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Proceedings of the International Conference on Learning Representations (ICLR). San Diego, USA (2015)

  3. Banchs, R.E., Li, H.: Iris: a chat-oriented dialogue system based on the vector space model. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 37–42. Jeju, Korea (2012)

  4. Chung, J., Gulcehre, C., Cho, K.H., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555 (2014)

  5. Ding, X., Liu, B., Yu, P.S.: A holistic lexicon-based approach to opinion mining. In: Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 231–240. Palo Alto, USA (2008)

  6. Gao, L., Guo, Z., Zhang, H., Xu, X., Shen, H.T.: Video captioning with attention-based lstm and semantic consistency. IEEE Trans. Multimed. 19(9), 2045–2055 (2017)

    Article  Google Scholar 

  7. Guo, Y., Zhang, J., Gao, L.: Exploiting long-term temporal dynamics for video captioning. World Wide Web (2018)

  8. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  9. Hu, M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 168–177. ACM, Sydney (2004)

  10. Huang, M., Cao, Y., Dong, C.: Modeling rich contexts for sentiment classification with lstm. arXiv:1605.01478 (2016)

  11. Inaba, M., Takahashi, K.: Neural utterance ranking model for conversational dialogue systems. In: Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 393– 403. Los Angeles, USA (2016)

  12. Ji, Y.L., Dernoncourt, F.: Sequential short-text classification with recurrent and convolutional neural networks. In: Proceedings of the conference of North American Chapter of Association for Computional Linguistics (NAACL), pp. 515–520. San Diego, USA (2016)

  13. Kadlec, R., Schmid, M., Bajgar, O., Kleindienst, J.: Text understanding with the attention sum reader network. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 908–918. Berlin, Germany (2016)

  14. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 655–665. Baltimore, USA (2014)

  15. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751. Doha, Qatar (2014)

  16. Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B.: A diversity-promoting objective function for neural conversation models. In: Proceedings of the Conference of North American Chapter of Association for Computional Linguistics (NAACL), pp. 110–119. San Diego, USA (2016)

  17. Li, J., Galley, M., Brockett, C., Spithourakis, G.P., Gao, J., Dolan, B.: A persona-based neural conversation model. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 994–1003. Berlin, Germany (2016)

  18. Li, X., Zhou, Z., Chen, L., Gao, L.: Residual attention-based lstm for video captioning. World Wide Web (2018)

  19. Lowe, R., Pow, N., Serban, I., Pineau, J.: The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In: Proceedings of the SIGDIAL 2015 Conference, pp. 285–294. Prague, Czech (2015)

  20. Luong, M.T., Sutskever, I., Le, Q.V., Vinyals, O., Zaremba, W.: Addressing the rare word problem in neural machine translation. Bulletin of University of Agricultural Sciences and Veterinary Medicine Cluj-Napoca Vet. Med. 27(2), 82–86 (2014)

    Google Scholar 

  21. Mei, H., Bansal, M., Walter, M.R.: Coherent dialogue with attention-based language models. In: Proceedings of the National Conference on Artificial Intelligence (AAAI), pp. 3252–3258. San Francisco, USA (2017)

  22. Misu, T., Georgila, K., Leuski, A., Traum, D.: Reinforcement learning of question-answering dialogue policies for virtual museum guides. In: Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 84–93. Seoual National University (2012)

  23. Mohammad, S.M., Kiritchenko, S., Zhu, X.: Nrc-canada: Building the state-of-the-art in sentiment analysis of tweets. In: 2nd Joint Conference on Lexical and Computational Semantics, pp. 321–327. Atlanda, USA (2013)

  24. Mrkšić, N., Séaghdha, D.O., Thomson, B., Gašić, M., Su, P.H., Vandyke, D., Wen, T.H., Young, S.: Multi-domain dialog state tracking using recurrent neural networks. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 794–799. Beijing, China (2015)

  25. Ritter, A., Cherry, C., Dolan, W.B.: Data-driven response generation in social media. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 583–593. Edinburgh, England (2011)

  26. Schatzmann, J., Weilhammer, K., Stuttle, M., Young, S.: A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. Knowl. Eng. Rev. 21(2), 97–126 (2006)

    Article  Google Scholar 

  27. Serban, I.V., Lowe, R., Charlin, L., Pineau, J.: Generative deep neural networks for dialogue: A short review. In: Advances in Neural Information Processing Systems, Workshop on Learning Methods for Dialogue. Barcelona, Spain (2016)

  28. Serban, I.V., Sordoni, A., Bengio, Y., Courville, A., Pineau, J.: Building end-to-end dialogue systems using generative hierarchical neural network models. In: Proceedings of the National Conference on Artificial Intelligence (AAAI), pp. 3776–3784. Phoenix, USA (2016)

  29. Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1577–1586. Beijing, China (2015)

  30. Song, J., Gao, L., Nie, F., Shen, H.T., Yan, Y., Sebe, N.: Optimized graph learning with partial tags and multiple features for image and video annotation. IEEE Trans. Image Process. 25(11), 4999–5011 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  31. Song, J., Gao, L., Liu, L., Zhu, X., Sebe, N.: Quantization-based hashing: A general framework for scalable image and video retrieval. Pattern Recog. 75, 175–187 (2017)

    Article  Google Scholar 

  32. Song, J., Guo, Z., Gao, L., Liu, W., Zhang, D., Shen, H.T.: Hierarchical lstm with adjusted temporal attention for video captioning. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI), pp. 2737–2743. Melbourne. Australia (2017)

  33. Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Nie, J.Y., Gao, J., Dolan, B.: A neural network approach to context-sensitive generation of conversational responses. In: Proceedings of the Conference of North American Chapter of Association for Computional Linguistics (NAACL), pp. 196–205. Danver, USA (2015)

  34. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112. Montreal, Canada (2014)

  35. Takahashi, T., Mera, K., Tang, B.N., Kurosawa, Y., Takezawa, T.: Natural language dialog system considering speakers emotion calculated from acoustic features. Dialogues with Social Robots, pp. 145–157 (2017)

  36. Teng, Z., Zhang, Y.: Bidirectional tree-structured lstm with head lexicalization. arXiv:1611.06788 (2016)

  37. Vinyals, O., Le, Q.: A neural conversational model. In: Proceedings of the International Conference on Machine Learning, Deep Learning Workshop. Lille, France (2015)

  38. Wang, X., Liu, Y., Sun, C., Wang, B., Wang, X.: Predicting polarities of tweets by composing word embeddings with long short-term memory. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1343–1353. Beijing, China (2015)

  39. Wen, T.H., Gasic, M., Mrksic, N., Su, P.H., Vandyke, D., Young, S.: Semantically conditioned lstm-based natural language generation for spoken dialogue systems. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1711–1721. Lisboa, Portugal (2015)

  40. Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Neural Language Processing, pp. 347–354. Vancouver, Canada (2005)

  41. Xia, H., Tao, M., Wang, Y.: Sentiment text classification of customers reviews on the Web based on svm. In: 6th International Conference on Natural Computation, pp. 3633–3637. Yantai, China (2010)

  42. Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., Bengio, Y.: Show, attend and tell: Neural image caption generation with visual attention. In: Proceeding of the International Conference on Machine Learning, pp. 2048–2057. Lille, France (2015)

  43. You, Q., Jin, H., Wang, Z., Fang, C., Luo, J.: Image captioning with semantic attention. In: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4651–4659. Las Vegas, USA (2016)

  44. Young, S., Keizer, S., Schatzmann, J., Thomson, B., Yu, K.: The hidden information state model: A practical framework for pomdp-based spoken dialogue management. Comput. Speech Lang. 24(2), 150–174 (2010)

    Article  Google Scholar 

  45. Zhang, L., Chen, C.: Sentiment classification with convolutional neural networks: An experimental study on a large-scale chinese conversation corpus. In: International Conference on Computational Intelligence and Security, pp. 165–169. Wuxi, China (2016)

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No.61672267 and No.61502208), the Open Project Programme of the National Laboratory of Pattern Recognition (NLPR, No.201700022), the general Financial Grant from the China Postdoctoral Science Foundation (No.2015M570413) and the Natural Science Foundation of Jiangsu Province (No.BK20140571).

Author information

Authors and Affiliations

  1. School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang, 212013, Jiangsu, China

    Feng Liu, Qirong Mao, Liangjun Wang, Nelson Ruwa, Jianping Gou & Yongzhao Zhan

Authors
  1. Feng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qirong Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liangjun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nelson Ruwa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianping Gou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yongzhao Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qirong Mao.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Deep vs. Shallow: Learning for Emerging Web-scale Data Computing and Applications

Guest Editors: Jingkuan Song, Shuqiang Jiang, Elisa Ricci, and Zi Huang

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, F., Mao, Q., Wang, L. et al. An emotion-based responding model for natural language conversation. World Wide Web 22, 843–861 (2019). https://doi.org/10.1007/s11280-018-0601-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-018-0601-2

Keywords

Search

Navigation