Abstract
Dialog state tracking (DST) is a core component in the task-oriented dialog systems. Many previous approaches regard DST as a classification task for a set of predefined slot-value pairs, but these approaches can not handle the situation of dynamic ontology. Other methods consider slot as span-based, while these methods are insufficient when the target slot-value cannot be found as a word segment in the dialog context. To mitigate these problems, we propose a Transformer-based model for DST. The proposed method can achieve the DST task by extracting the slot-value from the dialog context or classifying for the slot with limited values. Experimental evidence shows that our proposed model can achieve competitive performance on the WOZ 2.0 dataset while being 20\(\times \) faster than the state-of-the-art model. To demonstrate the effectiveness of our proposed model on multi-domain, we experiment with the recently released MultiWOZ-2.0 dataset. The experiment shows that our model can achieve promising result.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Budzianowski, P., et al.: MultiWOZ - a large-scale multi-domain wizard-of-Oz dataset for task-oriented dialogue modelling. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pp. 5016–5026 (2018). https://doi.org/10.18653/v1/D18-1547, https://www.aclweb.org/anthology/D18-1547
Chao, G.L., Lane, I.: BERT-DST: scalable end-to-end dialogue state tracking with bidirectional encoder representations from transformer. In: Proceedings of Interspeech 2019, pp. 1468–1472 (2019). https://doi.org/10.21437/Interspeech.2019-1355
Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: ELECTRA: pre-training text encoders as discriminators rather than generators. In: International Conference on Learning Representations (2020). https://openreview.net/pdf?id=r1xMH1BtvB
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186 (2019). https://doi.org/10.18653/v1/N19-1423, https://www.aclweb.org/anthology/N19-1423
Goel, R., Paul, S., Hakkani-Tür, D.: HyST: a hybrid approach for flexible and accurate dialogue state tracking. arXiv preprint arXiv:1907.00883 (2019)
Henderson, M., Thomson, B., Williams, J.D.: The second dialog state tracking challenge. In: Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pp. 263–272 (2014). https://doi.org/10.3115/v1/W14-4337, https://www.aclweb.org/anthology/W14-4337
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2015)
Lai, T.M., Hung Tran, Q., Bui, T., Kihara, D.: A simple but effective BERT model for dialog state tracking on resource-limited systems. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8034–8038 (2020). https://doi.org/10.1109/ICASSP40776.2020.9053975
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a lite BERT for self-supervised learning of language representations. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=H1eA7AEtvS
Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Mrkšić, N., Ó Séaghdha, D., Wen, T.H., Thomson, B., Young, S.: Neural belief tracker: data-driven dialogue state tracking. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 1777–1788 (2017). https://doi.org/10.18653/v1/P17-1163, https://www.aclweb.org/anthology/P17-1163
Nouri, E., Hosseini-Asl, E.: Toward scalable neural dialogue state tracking model. CoRR (2018). http://arxiv.org/abs/1812.00899
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+questions for machine comprehension of text. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383–2392 (2016). https://doi.org/10.18653/v1/D16-1264, https://www.aclweb.org/anthology/D16-1264
Ramadan, O., Budzianowski, P., Gašić, M.: Large-scale multi-domain belief tracking with knowledge sharing. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 432–437 (2018). https://doi.org/10.18653/v1/P18-2069, https://www.aclweb.org/anthology/P18-2069
Ren, L., Xie, K., Chen, L., Yu, K.: Towards universal dialogue state tracking. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2780–2786 (2018). https://doi.org/10.18653/v1/D18-1299, https://www.aclweb.org/anthology/D18-1299
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wen, T.H., et al.: A network-based end-to-end trainable task-oriented dialogue system. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pp. 438–449. Association for Computational Linguistics (2017). https://www.aclweb.org/anthology/E17-1042
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding. In: Advances in Neural Information Processing Systems, pp. 5754–5764 (2019)
Zhong, V., Xiong, C., Socher, R.: Global-locally self-attentive encoder for dialogue state tracking. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 1458–1467 (2018). https://doi.org/10.18653/v1/P18-1135, https://www.aclweb.org/anthology/P18-1135
Zhou, L., Small, K.: Multi-domain dialogue state tracking as dynamic knowledge graph enhanced question answering. arXiv preprint arXiv:1911.06192 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Miao, Y., Liu, K., Yang, W., Yang, C. (2022). A Novel Transformer-Based Model for Dialog State Tracking. In: Rau, PL.P. (eds) Cross-Cultural Design. Applications in Business, Communication, Health, Well-being, and Inclusiveness. HCII 2022. Lecture Notes in Computer Science, vol 13313. Springer, Cham. https://doi.org/10.1007/978-3-031-06050-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-06050-2_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06049-6
Online ISBN: 978-3-031-06050-2
eBook Packages: Computer ScienceComputer Science (R0)