Abstract
Dialogue state tracking (DST) is an important component in task-oriented dialogue systems. The task of DST is to identify or update the values of the given slots at every turn in the dialogue. Previous studies attempt to encode dialogue history into latent variables in the network. However, due to limited training data, it is valuable to encode prior knowledge that is available in different task-oriented dialogue scene. In this paper, we propose a neural network architecture to effectively incorporate prior knowledge into the encoding process. We performed experiment, in which entities belonging to the dialogue scene are extracted as the prior knowledge and are encoded along with the dialogue using the proposed architecture. Experiment results show significantly improvement in slot prediction accuracy, especially for slot types date and time, which are difficult to recognize by an encoder that is trained with limited data. Our results also achieve new state-of-the-art joint accuracy on the MultiWOZ 2.1 dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Budzianowski, P., et al.: Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 5016–5026 (2018)
Chen, J., Zhang, R., Mao, Y., Xu, J.: Parallel interactive networks for multi-domain dialogue state generation. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1921–1931. Association for Computational Linguistics, Online (November 2020). https://doi.org/10.18653/v1/2020.emnlp-main.151, https://www.aclweb.org/anthology/2020.emnlp-main.151
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186 (2019)
Eric, M., et al.: Multiwoz 2.1: a consolidated multi-domain dialogue dataset with state corrections and state tracking baselines (2019)
Goel, R., Paul, S., Hakkani-Tür, D.: Hyst: a hybrid approach for flexible and accurate dialogue state tracking. In: Proceedings of the Interspeech 2019, pp. 1458–1462 (2019)
Henderson, M., Thomson, B., Young, S.: Word-based dialog state tracking with recurrent neural networks. In: Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) (2014)
Hosseini-Asl, E., McCann, B., Wu, C.S., Yavuz, S., Socher, R.: A simple language model for task-oriented dialogue. arXiv preprint arXiv:2005.00796 (2020)
Kim, S., Yang, S., Kim, G., Lee, S.W.: Efficient dialogue state tracking by selectively overwriting memory. arXiv preprint arXiv:1911.03906 (2019)
Lee, H., Lee, J., Kim, T.Y.: Sumbt: Slot-utterance matching for universal and scalable belief tracking. arXiv preprint arXiv:1907.07421 (2019)
Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S.J., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Association for Computational Linguistics (ACL) System Demonstrations, pp. 55–60 (2014). http://www.aclweb.org/anthology/P/P14/P14-5010
Mehri, S., Eric, M., Hakkani-Tur, D.: Dialoglue: A natural language understanding benchmark for task-oriented dialogue. arXiv preprint arXiv:2009.13570 (2020)
Rastogi, A., Hakkani-Tur, D., Heck, L.: Scalable multi-domain dialogue state tracking. In: 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) (2017)
Shi, H., Ushio, T., Endo, M., Yamagami, K., Horii, N.: Convolutional neural networks for multi-topic dialog state tracking. In: Jokinen, K., Wilcock, G. (eds.) Dialogues with Social Robots. LNEE, vol. 999, pp. 451–463. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-2585-3_37
Thomson, B., Young, S.: Bayesian update of dialogue state: a pomdp framework for spoken dialogue systems. Comput. Speech Lang. 562–588 (2010)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wang, Z., Lemon, O.: A simple and generic belief tracking mechanism for the dialog state tracking challenge: on the believability of observed information. In: Proceedings of the SIGDIAL 2013 Conference, pp. 423–432 (2013)
Williams, J.D., Henderson, M., Raux, A., Thomson, B., Black, A., Ramachandran, D.: The dialog state tracking challenge series. AI Mag. 35(4), 121–124 (2014)
Yong, S., Li Zekang, Z.J., Meng Fandong, F.Y., Niu Cheng, Z.J.: A contextual hierarchical attention network with adaptive objective for dialogue state tracking. In: Proceedings of the 58th Conference of the Association for Computational Linguistics (2020)
Zeng, Y., Nie, J.Y.: Multi-domain dialogue state tracking based on state graph. arXiv preprint arXiv:2010.11137 (2020)
Zhang, J.G., et al.: Find or classify? dual strategy for slot-value predictions on multi-domain dialog state tracking. arXiv preprint arXiv:1910.03544 (2019)
Zhong, V., Xiong, C., Socher, R.: Global-locally self-attentive encoder for dialogue state tracking. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1458–1467 (2018)
Zhu, S., Li, J., Chen, L., Yu, K.: Efficient context and schema fusion networks for multi-domain dialogue state tracking. arXiv preprint arXiv:2004.03386 (2020)
Zilka, L., Jurcicek, F.: Incremental lstm-based dialog state tracker. In: 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (Asru), pp. 757–762. IEEE (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, Z., Liu, C. (2021). An Attention Method to Introduce Prior Knowledge in Dialogue State Tracking. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13110. Springer, Cham. https://doi.org/10.1007/978-3-030-92238-2_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-92238-2_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92237-5
Online ISBN: 978-3-030-92238-2
eBook Packages: Computer ScienceComputer Science (R0)