Abstract
This paper proposes a fully data-driven approach to dialog state tracking (DST) that can handle new slot values not seen during training. The approach is based on a long short-term memory recurrent neural network with an attention mechanism. Unlike with conventional attention mechanisms, we use encoded user utterances and a hypothesis for the slot value (the target value) to calculate attention weights. In addition, while conventional attention mechanisms focus on words that correspond to trained values, the proposed attention mechanism focuses on words that correspond to a target value. Therefore, the DST model can detect unseen values by adding values to the hypothesis as target values. The proposed approach is evaluated using the second and the third Dialog State Tracking Challenge datasets. Evaluation results show that the proposed method improves 10.3 points on unseen slot values.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Williams JD, Raux A, Henderson M (2016) The dialog state tracking challenge series: a review. Dialogue Discourse 7(3):4–33
Sun K, Chen L, Zhu S, Yu K (2014) A generalized rule based tracker for dialogue state tracking. In: Spoken language technology workshop (SLT). IEEE, pp 330–335
Kadlec R, Vodolan M, Libovickỳ J, Macek J, Kleindienst J (2014) Knowledge-based dialog state tracking. In: Spoken language technology workshop (SLT). IEEE, pp 348–353
Yoshino K, Hiraoka T, Neubig G, Nakamura S (2016) Dialogue state tracking using long short term memory neural networks. In: Proceedings of seventh international workshop on spoken dialog systems (IWSDS), pp 1–8
Zilka L, Jurcicek F (2015) Incremental LSTM-based dialog state tracker. In: Workshop on automatic speech recognition and understanding (ASRU). IEEE, pp 757–762
Liu B, Lane I (2017) An end-to-end trainable neural network model with belief tracking for task-oriented dialog. Proc Interspeech 2017:2506–2510
Henderson M, Thomson B, Young S (2014) Robust dialog state tracking using delexicalised recurrent neural networks and unsupervised adaptation. In: Spoken language technology workshop (SLT). IEEE, pp 360–365
Rastogi A, Hakkani-Tür D, Heck L (2017) Scalable multi-domain dialogue state tracking. In: Automatic speech recognition and understanding workshop (ASRU). IEEE, pp 561–568
Mrkšić N, Vulić I (2018) Fully statistical neural belief tracking. In: Proceedings of the 56th annual meeting of the association for computational linguistics, vol 2, pp 108–113
Xu P, Hu Q (2018) An end-to-end approach for handling unknown slot values in dialogue state tracking. In: Proceedings of the 56th annual meeting of the association for computational linguistics, vol 1, pp 1448–1457
Yoshida T, Iwata K, Fujimura H, Akamine M (2018) Dialog state tracking for unseen values using an extended attention mechanism. In: International workshop on spoken dialog system technology (IWSDS)
Chao G, Lane I (2019) BERT-DST: scalable end-to-end dialogue state tracking with bidirectional encoder representations from transformer. In: Proceedings of interspeech, pp 1468–1472
Wang J, Song Y, Leung T, Rosenberg C, Wang J, Philbin J, Chen B, Wu Y (2014) Learning fine-grained image similarity with deep ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1386–1393
Henderson M, Thomson B, Williams J (2014) The second dialog state tracking challenge. In 15th annual meeting of the special interest group on discourse and dialogue, vol 263
Henderson M, Thomson B, Williams JD (2014) The third dialog state tracking challenge. In: Spoken language technology workshop (SLT). IEEE, pp 324–329
Tokui S, Oono K, Hido S, Clayton J (2015) Chainer: a next-generation open source framework for deep learning. In: Proceedings of workshop on machine learning systems (LearningSys) in the twenty-ninth annual conference on neural information processing systems (NIPS)
Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Empirical methods in natural language processing (EMNLP), pp 1532–1543
Zhang W, Feng Y, Meng F, You D, Liu Q (2019) Bridging the gap between training and inference for neural machine translation. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 4334–4343
Acknowledgements
The authors would like to thank Professor Masami Akamine for his insightful comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Yoshida, T., Iwata, K., Kobayashi, Y., Fujimura, H. (2021). Dialog State Tracking with Incorporation of Target Values in Attention Models. In: D'Haro, L.F., Callejas, Z., Nakamura, S. (eds) Conversational Dialogue Systems for the Next Decade. Lecture Notes in Electrical Engineering, vol 704. Springer, Singapore. https://doi.org/10.1007/978-981-15-8395-7_9
Download citation
DOI: https://doi.org/10.1007/978-981-15-8395-7_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-8394-0
Online ISBN: 978-981-15-8395-7
eBook Packages: EngineeringEngineering (R0)