Abstract
In task-oriented dialogue systems, semantically controlled natural language generation is the procedure to generate responses based on current context information. Seq2seq models are widely used to generate dialogue responses and achieve favorable performance. Nevertheless, how to effectively obtain the dialogue’s key information from history remains to be a critical problem. To overcome this problem, we propose a Turn-level Recurrence Self-Attention (TRSA) encoder, which effectively obtains progressive structural relationship in turn-level from conversation history. Moreover, we propose a novel model to predict dialogue actions and generate dialogue responses jointly, which is different from the separate training model used in previous studies. Experiments demonstrate that our model alleviates the problem of inaccurate attention in dialogue history and improves the degree of dialogue completion significantly. In the large-scale MultiWOZ dataset, we improve the performance by 3.9% of inform rate and 3.4% of success rate, which is significantly higher than the state-of-the-art.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Budzianowski, P., et al.: Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. arXiv preprint arXiv:1810.00278 (2018)
Chen, W., Chen, J., Qin, P., Yan, X., Wang, W.Y.: Semantically conditioned dialog response generation via hierarchical disentangled self-attention. arXiv preprint arXiv:1905.12866 (2019)
Chen, X., Xu, J., Xu, B.: A working memory model for task-oriented dialog response generation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2687–2693 (2019)
Dai, Z., Yang, Z., Yang, Y., Cohen, W.W., Carbonell, J., Le, Q.V., Salakhutdinov, R.: Transformer-xl: attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860 (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Dušek, O., JurÄŤĂÄŤek, F.: A context-aware natural language generator for dialogue systems. arXiv preprint arXiv:1608.07076 (2016)
Gao, J., Galley, M., Li, L.: Neural approaches to conversational AI. In: Proceedings of ACL 2018, Tutorial Abstracts, pp. 2–7 (2018)
Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wen, T.H., et al.: Multi-domain neural network language generation for spoken dialogue systems. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 120–129 (2016)
Wen, T.H., Gasic, M., Mrksic, N., Su, P.H., Vandyke, D., Young, S.: Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. arXiv preprint arXiv:1508.01745 (2015)
Wen, T.H., et al.: A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562 (2016)
Acknowledgments
This work is supported in part by the National Key R&D Program of China(Grant No.2017YFB1400800). Engineering Research Center of Information Networks, Ministry of Education.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Tan, Y., Ou, Z., Liu, K., Shi, Y., Song, M. (2020). Turn-Level Recurrence Self-attention for Joint Dialogue Action Prediction and Response Generation. In: Wang, X., Zhang, R., Lee, YK., Sun, L., Moon, YS. (eds) Web and Big Data. APWeb-WAIM 2020. Lecture Notes in Computer Science(), vol 12318. Springer, Cham. https://doi.org/10.1007/978-3-030-60290-1_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-60290-1_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60289-5
Online ISBN: 978-3-030-60290-1
eBook Packages: Computer ScienceComputer Science (R0)