Abstract
There are many intents of dialogues that cannot be recognized due to the contextual features of conversation, resulting in service failure for online chatting robots. Current methods leverage memory networks or machine reading comprehension (MRC) for multi-turn conversation intent recognition. We proposed a novel model for dialogue intent recognition, which leverages the advantages of MRC and memory networks. The model uses a self-attention and co-attention based contextual flow block to aggregate the dialogue utterances for intent recognition. We built a Chinese multi-turn dialogue dataset and designed a multi-task learning method to train the model. The experiment results are exciting, where the proposed model gets 82.75% accuracy and 78.13% F1 score. Those results show promising feasibility to apply our method in online chatting robot.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
https://github.com/google/sentencepiece. We train a unigram based tokenizer in CMTD where the vocabulary size is 50K. The sentencepiece tokenizer splits a Chinese utterance into pieces and encode each piece with a unique integer.
References
Ahmad, W.U., Bai, X., Peng, N., Chang, K.W.: Learning robust, transferable sentence representations for text classification. arXiv preprint arXiv:1810.00681 (2018)
Bao, W., Wen, H., Li, S., Liu, X., Lin, Q., Yang, K.: GMCM: graph-based micro-behavior conversion model for post-click conversion rate estimation. In: Huang, J., et al. (eds.) Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2020, Virtual Event, China, 25–30 July 2020, pp. 2201–2210. ACM (2020)
Campos, J.A., Otegi, A., Soroa, A., Deriu, J., Cieliebak, M., Agirre, E.: DoQA-accessing domain-specific FAQs via conversational QA. arXiv preprint arXiv:2005.01328 (2020)
Chen, D., Fisch, A., Weston, J., Bordes, A.: Reading Wikipedia to answer open-domain questions. arXiv preprint arXiv:1704.00051 (2017)
Choi, E., et al.: QuAC: question answering in context. arXiv preprint arXiv:1808.07036 (2018)
Cui, Y., Che, W., Liu, T., Qin, B., Wang, S., Hu, G.: Revisiting pre-trained models for Chinese natural language processing. arXiv preprint arXiv:2004.13922 (2020)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186. Association for Computational Linguistics (2019)
Geva, M., Schuster, R., Berant, J., Levy, O.: Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913 (2020)
Hu, Z., et al.: Texar: a modularized, versatile, and extensible toolkit for text generation. In: ACL 2019, System Demonstrations (2019)
Kudo, T.: Subword regularization: improving neural network translation models with multiple subword candidates. In: Gurevych, I., Miyao, Y. (eds.) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 66–75. Association for Computational Linguistics (2018)
Kumar, A., et al.: Ask me anything: dynamic memory networks for natural language processing. In: International Conference on Machine Learning, pp. 1378–1387. PMLR (2016)
Liu, X., Zhang, Y., Liao, Y., Jiang, L.: Dynamic updating of the knowledge base for a large-scale question answering system. ACM Trans. Asian Low Resour. Lang. Inf. Process. 19(3), 45:1–45:13 (2020). https://doi.org/10.1145/3377708
Ma, X., Hovy, E.H.: End-to-end sequence labeling via bi-directional LATM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Volume 1: Long Papers, Berlin, Germany, 7–12 August 2016. The Association for Computer Linguistics (2016)
Qu, C., Yang, L., Qiu, M., Croft, W.B., Zhang, Y., Iyyer, M.: Bert with history answer embedding for conversational question answering. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1133–1136 (2019)
Rajendran, J., Ganhotra, J., Singh, S., Polymenakos, L.: Learning end-to-end goal-oriented dialog with multiple answers. arXiv preprint arXiv:1808.09996 (2018)
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)
Ravuri, S.V., Stolcke, A.: Recurrent neural network and LSTM models for lexical utterance classification. In: 16th Annual Conference of the International Speech Communication Association, INTERSPEECH 2015, Dresden, Germany, 6–10 September 2015, pp. 135–139. ISCA (2015)
Reddy, S., Chen, D., Manning, C.D.: CoQA: a conversational question answering challenge. Trans. Assoc. Comput. Linguist. 7, 249–266 (2019)
Shao, Z., Li, X., Guo, Y., Zhang, L.: Influence of service quality in sharing economy: Understanding customers’ continuance intention of bicycle sharing. Electron. Commer. Res. Appl. 40, 100944 (2020)
Sukhbaatar, S., Szlam, A., Weston, J., Fergus, R.: End-to-end memory networks. arXiv preprint arXiv:1503.08895 (2015)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA, 4–9 December 2017, pp. 5998–6008 (2017)
Weston, J., Chopra, S., Bordes, A.: Memory networks. arXiv preprint arXiv:1410.3916 (2014)
Xu, K., Lai, Y., Feng, Y., Wang, Z.: Enhancing key-value memory neural networks for knowledge based question answering. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2937–2947 (2019)
You, Y., Li, J., Hseu, J., Song, X., Demmel, J., Hsieh, C.: Reducing BERT pre-training time from 3 days to 76 minutes. CoRR abs/1904.00962 (2019)
Zhang, W., et al.: Large-scale causal approaches to debiasing post-click conversion rate estimation with multi-task learning. In: The Web Conference 2020, WWW 2020, Taipei, Taiwan, 20–24 April 2020, pp. 2775–2781. ACM/IW3C2 (2020)
Zhang, Y., Ou, Z., Yu, Z.: Task-oriented dialog systems that consider multiple appropriate responses under the same context. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 9604–9611 (2020)
Zhang, Z., Li, J., Zhu, P., Zhao, H., Liu, G.: Modeling multi-turn conversation with deep utterance aggregation. arXiv preprint arXiv:1806.09102 (2018)
Zhou, P., Qi, Z., Zheng, S., Xu, J., Bao, H., Xu, B.: Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling. In: Proceedings of the 26th International Conference on Computational Linguistics, COLING 2016, Technical Papers, Osaka, Japan, 11–16 December 2016, pp. 3485–3495. ACL (2016)
Acknowledgment
This work is funded by the XiaduoAI company and the customer intent recognition model is now applied to the shopping dialog robot of XiaoduoAI.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Z., Guo, T., Jiang, L., Gu, M. (2021). A Dialogue Contextual Flow Model for Utterance Intent Recognition in Multi-turn Online Conversation. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management . KSEM 2021. Lecture Notes in Computer Science(), vol 12816. Springer, Cham. https://doi.org/10.1007/978-3-030-82147-0_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-82147-0_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82146-3
Online ISBN: 978-3-030-82147-0
eBook Packages: Computer ScienceComputer Science (R0)