Abstract
Spoken language understanding plays an important role in the dialogue systems, and in such systems, intent detection and slot filling tasks are used to extract semantic components. In previous works on spoken language understanding, many ways that from traditional pipeline methods to joint models have been investigated. The features from these methods are usually extracted from one dataset that cannot jointly optimize databases with different distributions. In this paper, we propose a new adversarial shared-private attention network that learns features from two different datasets with shared and private spaces. The proposed adversarial network trains the shared attention network so that the shared distributions of two datasets are close, thereby reducing the redundancy of the shared features, which helps to alleviate the interference from the private and shared space. A joint training strategy between intent detection and slot filling is also applied to enhance the task relationship. Experimental results on public benchmark corpora, called ATIS, Snips and MIT, show that our proposed models significantly outperform other methods on intent accuracy, slot F1 measure and sentence accuracy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Tur, G., De Mori, R.: Spoken Language Understanding: Systems for Extracting Semantic Information From Speech. Wiley, Hoboken (2011)
Tur, G., Hakkani-Tür, D., Heck, L.: What is left to be understood in ATIS. In: IEEE Spoken Language Technology Workshop, pp. 19–24. IEEE (2010)
Hakkani-Tür, D., et al.: Multi-domain joint semantic frame parsing using bi-directional RNN-LSTM. In: Interspeech, pp. 715–719 (2016)
Liu, B., Lane, I.: Attention-based recurrent neural network models for joint intent detection and slot filling. In: Interspeech, pp. 685–689 (2016)
Goo, C.-W., et al.: Slot-gated modeling for joint slot filling and intent prediction. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 753–757 (2018)
Liu, P., Qiu, X., Huang, X.: Deep multi-task learning with shared memory for text classification. In: EMNLP, pp. 118–127 (2016)
Liu, B., Lane, I.: Multi-domain adversarial learning for slot filling in spoken language understanding. arXiv preprint arXiv:1711.11310 (2017)
Lan, O., Zhu, S., Yu, K.: Semi-supervised training using adversarial multi-task learning for spoken language understanding. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6049–6053 (2018)
Masumura, R., Shinohara, Y., Higashinaka, R., et al.: Adversarial training for multi-task and multi-lingual joint modeling of utterance intent classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 633–639 (2018)
Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17, 2096–2030 (2016)
Tzeng, E., Hoffman, J., Saenko, K., Darrell, T.: Adversarial discriminative domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7167–7176 (2017)
Hochreiter, S., Schmidhuber, J.: Long short- term memory. Neural Comput. 9(8), 1735–1780 (1997)
Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Raymond, C., Riccardi, G.: Generative and discriminative algorithms for spoken language understanding. In: Interspeech, pp. 1605–1608 (2007)
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 1180–1189 (2015)
Coucke, A., et al.: Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint arXiv:1805.10190 (2018)
Goldberg, Y., Levy, O.: word2vec explained: deriving Mikolov et al’.s negative-sampling word-embedding method. arXiv preprint arXiv:1402.3722 (2014)
Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Lechevallier, Y., Saporta, G. (eds.) Proceedings of COMPSTAT 2010, pp. 177–186. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-7908-2604-3_16
Haihong, E., Niu, P., Chen, Z., Song, M.: A novel bi-directional interrelated model for joint intent detection and slot filling. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5467–5471 (2019)
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant 61771333, the Tianjin Municipal Science and Technology Project under Grant 18ZXZNGX00330.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, M., Wang, L., Si, Y., Dang, J. (2020). Adversarial Shared-Private Attention Network for Joint Slot Filling and Intent Detection. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1332. Springer, Cham. https://doi.org/10.1007/978-3-030-63820-7_71
Download citation
DOI: https://doi.org/10.1007/978-3-030-63820-7_71
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63819-1
Online ISBN: 978-3-030-63820-7
eBook Packages: Computer ScienceComputer Science (R0)