Abstract
There are two difficulties in existing spoken language understanding models. The first problem is that it is difficult to extract the implicit relationship information between the intention and the slot in the utterance for the inference process, and the inference effect is not ideal; the second problem is that the training data is scarce, and the existing models cannot learn from the small amount of training data. Get more useful information. To address these two challenges, this paper proposes an Explicit-Memory Few-shot join learning model. In order to solve the first problem, a multi-layer model structure from coarse to fine is adopted to train the hidden semantic relationship and hidden state information between intentions and slots in the utterance; in order to solve the second problem, using the Siamese BERT metric learning method to jointly train the model. We use the Snips and ATIS datasets to train the model, and the test results show better results. In the case of a small amount of data, the model can also obtain stronger inference ability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Guo, D., Tur, G., Yih, W.-T., Zweig, G.: Joint semantic utterance classification and slot filling with recursive neural networks. In: 2014 IEEE Spoken Language Technology Workshop (SLT), South Lake Tahoe, NV, pp. 554–559. IEEE (2014)
Hakkani-Tür, D., et al.: Multi-domain joint semantic frame parsing using bi-directional RNN-LSTM. In: Interspeech 2016, 17th Annual Conference of the International Speech Communication Association, San Francisco, CA, pp. 715–719. ISCA (2016)
Liu, B., Lane, I.: Attention-based recurrent neural network models for joint intent detection and slot filling. In: Interspeech 2016, 17th Annual Conference of the International Speech Communication Association, San Francisco, CA, pp. 685–689. ISCA (2016)
Goo, C.-W., et al.: Slot-gated modeling for joint slot filling and intent prediction. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, vol. 2 (Short Papers), pp. 753–757. Association for Computational Linguistics (2018)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint (2014)
Liu, H., Zhang, F., Zhang, X., Zhao, S., Zhang, X.: An explicit-joint and supervised-contrastive learning framework for few-shot intent classification and slot filling. arXiv preprint (2021)
Kumar, M., Kumar, V., Glaude, H., de Lichy, C., Alok, A., Gupta, R.: Protoda: efficient transfer learning for few-shot intent classification. In: 2021 IEEE Spoken Language Technology Workshop (SLT), Shenzhen, China, pp. 966–972. IEEE (2021)
Zhang, J., et al.: Few-shot intent detection via contrastive pre-training and fine-tuning. In: 2021 EMNLP Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 1906–1912 (2021)
Su, J., Weijie, L., Yangyiwen, O.: Whitening sentence representations for better semantics and faster retrieval. arXiv preprint (2021)
Zhang, H., et al.: Fine-tuning pre-trained language models for few-shot intent detection: supervised pre-training and isotropization. arXiv preprint (2022)
Hashemi, H.B., Asiaee, A., Kraft, R.: Query intent detection using convolutional neural networks. In: International Conference on Web Search and Data Mining, Workshop on Query Understanding (2016)
Bhargava, A., Celikyilmaz, A., Hakkani-Tür, D., Sarikaya, R.: Easy contextual intent prediction and slot detection. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, pp. 8337–8341. IEEE (2013)
Ravuri, S., Stolcke, A.: Recurrent neural network and LSTM models for lexical utterance classification. In: Interspeech 2015, 16th Annual Conference of the International Speech Communication Association, Dresden, Germany, pp. 135–139. ISCA (2015)
Li, C., Li, L., Qi, J.: A self-attentive model with gate mechanism for spoken language understanding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 3824–3833. Association for Computational Linguistics (2018)
Zhang, C., Li, Y., Du, N., Fan, W., Yu, P.: Joint slot filling and intent detection via capsule neural networks. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 5259–5267. Association for Computational Linguistics (2019)
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese BERT-networks. arXiv preprint (2019)
Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 5998–6008. Curran Associates Inc. (2017)
Coucke, A., et al.: Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint (2018)
Tur, G., Hakkani-Tür, D., Heck, L.: What is left to be understood in ATIS? In: 2010 IEEE Spoken Language Technology Workshop, Berkeley, CA, pp. 19–24. IEEE (2010)
Niu, P., Chen, Z., Song, M.: A novel bi-directional interrelated model for joint intent detection and slot filling. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 5467–5471. Association for Computational Linguistics (2019)
Chen, Q., Zhuo, Z., Wang, W.: BERT for joint intent classification and slot filling. arXiv preprint (2019)
Lewis, M., et al.: Bart: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Du, F., Liu, M., Zhao, T., Ail, S. (2023). An Explicit-Memory Few-Shot Joint Learning Model. In: Liu, F., Duan, N., Xu, Q., Hong, Y. (eds) Natural Language Processing and Chinese Computing. NLPCC 2023. Lecture Notes in Computer Science(), vol 14302. Springer, Cham. https://doi.org/10.1007/978-3-031-44693-1_62
Download citation
DOI: https://doi.org/10.1007/978-3-031-44693-1_62
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44692-4
Online ISBN: 978-3-031-44693-1
eBook Packages: Computer ScienceComputer Science (R0)
