Abstract
Natural language understanding (NLU) aims to extract schematic information contained in user utterances, which allows down streaming module of dialogue system, i.e., Dialogue Manager (DM) to process user queries and serve users in accomplishing their goal. If NLU component detects information improperly, it will cause error propagation and failure of all subsequent modules. Although the development of an adequate conversation system is challenging because of its periodic and contextual nature, its efficacy, applicability, and positive impact continue to fuel its recent surge and attention in the research community. The proposed work is the first of its kind, which attempts to develop a unified, multitasking, and context-aware BERT-based model for all NLU tasks, i.e., Domain classification (DC), Intent detection (ID), Slot filling (SF). Additionally, we have also incorporated a zero-shot intent detection technique in our proposed model for dealing with new and emerging intents effectively. The experimental results, as well as comparisons to the present state-of-the-art model and other several baselines on a benchmark dataset, firmly establish the efficacy and necessity of the proposed model.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Chen, Q., Zhuo, Z., Wang, W.: Bert for joint intent classification and slot filling. arXiv preprint arXiv:1902.10909 (2019)
Chen, S., Wais, S.Y.: Word attention for joint intent detection and slot filling. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 9927–9928 (2019)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, 2–7 June 2019, Volume 1 (Long and Short Papers), pp. 4171–4186 (2019)
Gupta, A., Zhang, P., Lalwani, G., Diab, M.: CASA-NLU: context-aware self-attentive natural language understanding for task-oriented chatbots. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China. Association for Computational Linguistics, November 2019
Kim, Y.-B., Lee, S., Stratos, K.: OneNet: joint domain, intent, slot prediction for spoken language understanding. In: 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 547–553. IEEE (2017)
Kim, Y.-B., Stratos, K., Sarikaya, R.: Scalable semi-supervised query classification using matrix sketching. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 8–13 (2016)
Liu, B., Lane, I.: Attention-based recurrent neural network models for joint intent detection and slot filling. In: Morgan, N. (eds.) Interspeech 2016, 17th Annual Conference of the International Speech Communication Association, San Francisco, CA, USA, 8–12 September 2016, pp. 685–689. ISCA (2016)
Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 1412–1421. Association for Computational Linguistics, September 2015
Peskov, D., et al.: Multi-domain goal-oriented dialogues (MultiDoGO): strategies toward curating and annotating large scale dialogue data. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China. Association for Computational Linguistics, November 2019
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates Inc. (2017)
Sarikaya, R., et al.: An overview of end-to-end language understanding and dialog management for personal digital assistants. In: 2016 IEEE Spoken Language Technology Workshop (SLT), pp. 391–397. IEEE (2016)
Sutton, C., McCallum, A.: An introduction to conditional random fields (2010)
Xia, C., Zhang, C., Yan, X., Chang, Y., Yu, P.: Zero-shot user intent detection via capsule neural networks. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium. Association for Computational Linguistics, October–November 2018
Xiao, L., Zhang, H., Chen, W., Wang, Y., Jin, Y.: MCapsNet: capsule network for text with multi-task learning. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4565–4574 (2018)
Acknowledgement
Dr. Sriparna Saha gratefully acknowledges the Young Faculty Research Fellowship (YFRF) Award, supported by Visvesvaraya Ph.D. Scheme for Electronics and IT, Ministry of Electronics and Information Technology (MeitY), Government of India, being implemented by Digital India Corporation (formerly Media Lab Asia) for carrying out this research.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Priya, N., Tiwari, A., Saha, S. (2021). Context Aware Joint Modeling of Domain Classification, Intent Detection and Slot Filling with Zero-Shot Intent Detection Approach. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13110. Springer, Cham. https://doi.org/10.1007/978-3-030-92238-2_48
Download citation
DOI: https://doi.org/10.1007/978-3-030-92238-2_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92237-5
Online ISBN: 978-3-030-92238-2
eBook Packages: Computer ScienceComputer Science (R0)