Abstract
This is an extension from a selected paper from JSAI2020. In order to generate more content-rich responses, as well as avoid irrelevant or monotonous ones, some researchers set off to focus on dialogue generation models in which causal relation is also taken into consideration. However, it is hard to distinguish whether there is causal relation between speaker’s and responder’s utterances from dialogue automatically, which makes collecting such utterance pairs even harder. In this paper, we first propose a transfer learning method to learn causality features from web contents such as Wikipedia dataset and pre-train a causality classifier model. Subsequently, the causality classifier model is taken advantages into collecting utterance pairs containing causality, which is used in the re-ranking method of seq2seq generation model instead of rule-based one, to improve the dialogue diversity and context coherency. An evaluation is conducted to show that the proposed method improves diversity and coherency of dialogues generation from three aspects: 1. The causality classifier model used for detecting dialogue pairs is confirmed by three famous word embedding methods and two famous corpus. 2. The causality generation model improves response diversity, which is evaluated by automatic dialogue evaluation metrics. 3. The causality generation model improves dialogue context coherency, which is evaluated by human observes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the Empirical Methods in Natural Language Proceedings (EMNLP) Conference (2014)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Neural Information Processing Systems (NeurlPS) Conference (2014)
Serban, I.V., Sordoni, A., Bengio, Y., Courville, A., Pineau, J.: Building end-to-end dialogue systems using generative hierarchical neural network models. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI-16) (2016)
Colombo, P., Witon, W., Modi, A., Kennedy, J., Kapadia, M.: Affect-driven dialog generation. CoRR (2019)
Serban, I.V., et al.: A hierarchical latent variable encoder-decoder model for generating dialogues. In: Proceedings of the 31th AAAI Conference on Artificial Intelligence (AAAI-17) (2017)
Sato, S., Inoue, N., Inui, K., Higuchi, S., Watanabe, N.: Towards generating non-task-oriented conversational responses using causal knowledge base. In: The 30th Annual Conference of the Japanese Society for Artificial Intelligence (JSAI2016) (2016)
Tanaka, S., Yoshino, K., Sudoh,K., Nakamura, S.: Conversational response re-ranking based on event causality and role factored tensor event embedding. In: 1st Workshop NLP for Conversational AI (ConvAI), ACL 2019 Workshop (2019)
Blanco, E., Castell, N., Moldovan, D.: Causal relation extraction. In: Proceedings of the International Conference on Language Resources and Evaluation (2008)
Kyriakakis, M., Androutsopoulos, I., Ametlle, J.G., Saudabaye, A.: Transfer learning for causal sentence detection. In: Proceedings of the 18th BioNLP Workshop and Shared Task (2019)
Peters, M.E., et al.: Deep contextualized word representations. In: Proceeding of the Conference of the NAACL: Human Language Technologies (2018)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Hendricks, I., et al.: SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions (2018)
Mirza, P., Sprugnoli, R., Tonelli, S., Speranza, M.: Annotating causality in the TempEval-3 corpus. In: Proceeding of the EACL Workshop on Computational Approaches to Causality in Language (2014)
Caselli, T., Vossen, P.: The event storyline corpus: A new benchmark for causal and temporal relation extraction. In: Proceedings of the Events and Stories in the News Workshop (2014)
Li, J., Monroe, W., Jurafsky, D.: A simple, fast diverse decoding algorithm for neural conversation models. arXiv preprint arXiv:1611.08562 (2016)
Danescu-Niculescu-Mizil, C., Lee, L.: Chameleons in imagined conversations: a new approach to understanding coordination of linguistic style in dialogs. In: Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics. ACL (2011)
Yang, B., Wu, J., Hattori, G.: A transfer learning method of data collection for dialogue response generation concerning causal relation. In: The 34th Annual Conference of the Japanese Society for Artificial Intelligence (JSAI2020) (2020)
Tanaka, S., Yoshino, K., Sudoh, K., Nakamura, S.: Conversational response re-ranking based on event causality and role factored tensor event embedding. In: Proceeding of the First Workshop on NLP for Conversational AI (2019)
Ryoko, T., Ryuta, T.: An analysis of ‘distinctive’ utterances in non-task-oriented conversational dialogue. Trans. Jpn. Soc. Artif. Intell. 22, 425–435 (2007)
Asghar, N.: automatic extraction of causal relations from natural language texts: a comprehensive survey. In: Proceeding of the 19th Annual SIGdial Meeting on Discourse and Dialogue (2016)
Luo, Z., Sha, Y., Zhu, K.Q., Hwang, S., Wang, Z.: Commonsense causal reasoning between short texts. In: Proceedings of Fifteenth International Conference on Principles of Knowledge Representation and Reasoning (KR2016) (2016)
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2016)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv:1301.3781 (2013)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI 1, 9 (2019)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014)
Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B.: A diversity-promoting objective function for neural conversation models. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2016)
Ram, A., et al.: Conversational AI: the science behind the alexa prize. In: 1st Proceedings of Alexa Prize (Alexa Prize 2017) (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, B., Wu, J., Hattori, G. (2021). Context-Aware Dialogue Response Generation Integrating Causal Relation. In: Yada, K., et al. Advances in Artificial Intelligence. JSAI 2020. Advances in Intelligent Systems and Computing, vol 1357. Springer, Cham. https://doi.org/10.1007/978-3-030-73113-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-73113-7_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73112-0
Online ISBN: 978-3-030-73113-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)