Abstract
To alleviate the ambiguity of user utterances, a profile-based spoken language understanding (PROSLU) task has been proposed recently, which uses supporting profile information as supplementary knowledge. The knowledge graph, as one of the supporting profile information, contains many entities with rich attribute information which do help the performance of SLU. However, existing models treat all entities as useful, resulting in a lot of noise being fed. Moreover, not all the attribute information of the entity is necessary, and there is a lot of noise and redundancy. In this paper, we propose a Noise-Removal of Knowledge Graph framework for PROSLU (NRKG-PROSLU), with two different kinds of denoising. One is a hard denoising by entity selection, where we introduce a small clean dataset and propose an auxiliary model, BERT-based entity selection, to filter out entities irrelevant to user utterances, namely noisy entities. The other is a soft denoising by entity attribute information selection, where we introduce keywords-based local semantic selection, which uses keywords to give more weight to relevant local semantics, so as to capture task-related information in the selected entities for reducing noise and redundancy. Then, the denoised knowledge graph is used to assist the SLU model, which achieves better performance than competitive models on the public PROSLU dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tur, G., De Mori, R.: Spoken Language Understanding: Systems for Extracting Semantic Information from Speech. Wiley, New York (2011)
Yao, K., Peng, B., Zweig, G., Yu, D., Li, X., Gao, F.: Recurrent conditional random field for language understanding. In: Proceedings of ICASSP 2014, Florence, Italy, 4–9 May 2014, pp. 4077–4081 (2014)
Yao, K., Peng, B., Zhang, Y., Yu, D., Zweig, G., Shi, Y.: Spoken language understanding using long short-term memory neural networks. In: Proceedings of SLT 2014, South Lake Tahoe, NV, USA, 7–10 December 2014, pp. 189–194 (2014)
Liu, B., Lane, I.R.: Attention-based recurrent neural network models for joint intent detection and slot filling. In: Proceedings of INTERSPEECH 2016, San Francisco, CA, USA, 8–12 September 2016, pp. 685–689 (2016)
Hakkani-Tür, D.: Multi-domain joint semantic frame parsing using bi-directional RNN-LSTM. In: Proceedings of INTERSPEECH 2016, San Francisco, CA, USA, 8–12 September 2016, pp. 715–719 (2016)
Zhang, X., Wang, H.: A joint model of intent determination and slot filling for spoken language understanding. In: Proceedings of IJCAI 2016, New York, NY, USA, 9–15 July 2016, pp. 2993–2999 (2016)
Goo, C.-W.: Slot-gated modeling for joint slot filling and intent prediction. In: Proceedings of NAACL-HLT 2018, New Orleans, Louisiana, USA, 1–6 June 2018, pp. 753–757 (2018)
Wang, Y., Shen, Y., Jin, H.: A bi-model based RNN semantic frame parsing model for intent detection and slot filling. In: Proceedings of NAACL-HLT 2018, New Orleans, Louisiana, USA, 1–6 June 2018, pp. 309–314 (2018)
Li, C., Li, L., Qi, J.: A self-attentive model with gate mechanism for spoken language understanding. In: Proceedings of EMNLP 2018, Brussels, Belgium, 31 October–4 November 2018, pp. 3824–3833 (2018)
Haihong, E., Niu, P., Chen, Z., Song, M.: A novel bi-directional interrelated model for joint intent detection and slot filling. In: Proceedings of ACL 2019, Florence, Italy, 28 July–2 August 2019, pp. 5467–5471 (2019)
Zhu, Z., Huang, P., Huang, H., Liu, S., Lao, L.: A graph attention interactive refine framework with contextual regularization for jointing intent detection and slot filling. In: Proceedings of ICASSP 2022, Virtual and Singapore, 23–27 May 2022, pp. 7617–7621 (2022)
Qin, L., Che, W., Li, Y., Wen, H., Liu, T.: A stack-propagation framework with token-level intent detection for spoken language understanding. In: Proceedings of EMNLP-IJCNLP 2019, Hong Kong, China, 3–7 November 2019, pp. 2078–2087 (2019)
Xu, X., Qin, L., Chen, K., Wu, G., Li, L., Che, W.: Text is no more enough! A benchmark for profile-based spoken language understanding. In: Proceedings of AAAI 2022, Vancouver, BC, Canada, 22 February–1 March 2022, pp. 11575–11585 (2022)
Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: Proceedings of EMNLP 2015, Lisbon, Portugal, 17–21 September 2015, pp. 1412–1421 (2015)
Libovický, J., Helcl, J.: Attention strategies for multi-source sequence-to-sequence learning. In: Proceedings of ACL 2017, Vancouver, Canada, 30 July–4 August 2017, pp. 196–202 (2017)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Vaswani, A.: Attention is all you need. In: Proceedings of NIPS 2017, Long Beach, CA, USA, 4–9 December 2017, pp. 5998–6008 (2017)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT 2019, Minneapolis, MN, USA, 2–7 June 2019, pp. 4171–4186 (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of ICLR 2015, San Diego, CA, USA, 7–9 May 2015
Liu, Y.: RoBERTa: a robustly optimized BERT pretraining approach. CoRR, abs/1907.11692 (2019)
Clark, K., Luong, M.-T., Le, Q.V., Manning, C.D.: ELECTRA: pre-training text encoders as discriminators rather than generators. In: Proceedings of ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020
Acknowledgements
This work was supported by Natural Science Foundation of Guangdong Province (No. 2021A1515011864) and Special Funds for the Cultivation of Guangdong College Students’ Scientific and Technological Innovation (No. pdjh2022a007).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lao, L., Huang, P., Zhu, Z., Liu, H., Lian, P., Xu, Y. (2023). A Noise-Removal of Knowledge Graph Framework for Profile-Based Spoken Language Understanding. In: Liu, F., Duan, N., Xu, Q., Hong, Y. (eds) Natural Language Processing and Chinese Computing. NLPCC 2023. Lecture Notes in Computer Science(), vol 14302. Springer, Cham. https://doi.org/10.1007/978-3-031-44693-1_63
Download citation
DOI: https://doi.org/10.1007/978-3-031-44693-1_63
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44692-4
Online ISBN: 978-3-031-44693-1
eBook Packages: Computer ScienceComputer Science (R0)
