Abstract
Open intent detection aims to correctly classify known intents and identify unknown intents that never appear in training samples, thus it is of practical importance in dialogue systems. Discriminative intent representation learning is a key challenge of open intent detection. Previous methods usually restrict known intent features to compact regions to learn the representations, which assumes that open intent is outside regions. However, open intent can be distributed among known intents. To address this issue, this paper proposes a triplet-contrastive learning strategy to learn discriminative semantic representations and differentiate between similar open intents and known intents. Further, a method named Triplet-Contrastive Adaptive Boundary (TCAB) is proposed, which leverages the triplet-contrastive learning strategy and an adaptive decision boundary method to detect open intent. Extensive experiments on three benchmark datasets show that our method achieves substantial improvements compared with a list of baseline methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Weld, H., Huang, X., Long, S., Poon, J., Han, S.C.: A survey of joint intent detection and slot filling models in natural language understanding. ACM Comput. Surv. (CSUR) (2021)
Liu, H., Liu, Y., Wong, L.P., Lee, L.K., Hao, T.: A hybrid neural network BERT-cap based on pre-trained language model and capsule network for user intent classification. In: Complexity 2020 (2020)
Luo, Y., Huang, Z., Wong, L.P., Zhan, C., Wang, F.L., Hao, T.: An early prediction and label smoothing alignment strategy for user intent classification of medical queries. In: International Conference on Neural Computing for Advanced Applications, pp. 115–128 (2022)
Liu, Y., Hao, T., Liu, H., Mu, Y., Weng, H., Wang, F.L.: OdeBERT: one-stage deep-supervised early-exiting BERT for fast inference in user intent classification. ACM Trans. Asian Low-Resource Lang. Inf. Process. 22(5), 1–18 (2023)
Hao, T., Li, X., He, Y., Wang, F.L., Qu, Y.: Recent progress in leveraging deep learning methods for question answering. In: Neural Computing and Applications, pp. 1–19 (2022)
Lin, T.E., Xu, H.: Deep unknown intent detection with margin loss. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 5491–5496 (2019)
Xu, H., He, K., Yan, Y., Liu, S., Liu, Z., Xu, W.: A deep generative distance-based classifier for out-of-domain detection with mahalanobis space. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 1452–1460 (2020)
Zhan, L.M., Liang, H., Liu, B., Fan, L., Wu, X.M., Lam, A.Y.: Out-of-scope intent detection with self-supervision and discriminative training. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 3521–3532 (2021)
Zhou, W., Liu, F., Chen, M.: Contrastive out-of-distribution detection for pretrained transformers. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 1100–1111 (2021)
Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016)
Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LoF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104 (2000)
Zhang, H., Xu, H., Lin, T.E.: Deep open intent classification with adaptive decision boundary. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 14374–14382 (2021)
Cheng, Z., Jiang, Z., Yin, Y., Wang, C., Gu, Q.: Learning to classify open intent via soft labeling and manifold mixup. IEEE/ACM Trans. Audio Speech Lang. Process. 30, 635–645 (2022)
Shu, L., Benajiba, Y., Mansour, S., Zhang, Y.: Odist: open world classification via distributionally shifted instances. In: Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 3751–3756 (2021)
Ryu, S., Koo, S., Yu, H., Lee, G.G.: Out-of-domain detection based on generative adversarial network. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 714–718 (2018)
Zheng, Y., Chen, G., Huang, M.: Out-of-domain detection for natural language understanding in dialog systems. IEEE/ACM Trans. Audio Speech Lang. Process. 28, 1198–1209 (2020)
Choi, D., Shin, M.C., Kim, E., Shin, D.R.: Outflip: generating examples for unknown intent detection with natural language attack. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 504–512 (2021)
Shu, L., Xu, H., Liu, B.: Doc: deep open classification of text documents. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2911–2916 (2017)
Yan, G., et al.: Unknown intent detection using gaussian mixture model with an application to zero-shot intent classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1050–1060 (2020)
Zeng, Z., et a.: Modeling discriminative representations for out-of-domain detection with supervised contrastive learning. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pp. 870–878 (2021)
Zhang, H., Xu, H., Zhao, S., Zhou, Q.: Towards open intent detection. arXiv preprint arXiv:2203.05823 (2022)
Kenton, J.D.M.W.C., Toutanova, L.K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)
Lin, T.E., Xu, H., Zhang, H.: Discovering new intents via constrained deep adaptive clustering with cluster refinement. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 8360–8367 (2020)
Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)
Scheirer, W.J., de Rezende Rocha, A., Sapkota, A., Boult, T.E.: Toward open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1757–1772 (2012)
Casanueva, I., Temčinas, T., Gerz, D., Henderson, M., Vulić, I.: Efficient intent detection with dual sentence encoders. In: Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pp. 38–45 (2020)
Larson, S., et al.: An evaluation dataset for intent classification and out-of-scope prediction. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1311–1316 (2019)
Coucke, A., et al.: Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint arXiv:1805.10190 (2018)
Bendale, A., Boult, T.E.: Towards open set deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1563–1572 (2016)
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Acknowledgements
The work described in this paper was substantially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (UGC/FDS16/E09/22).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chen, G. et al. (2023). A Triplet-Contrastive Representation Learning Strategy for Open Intent Detection. In: Zhang, H., et al. International Conference on Neural Computing for Advanced Applications. NCAA 2023. Communications in Computer and Information Science, vol 1870. Springer, Singapore. https://doi.org/10.1007/978-981-99-5847-4_17
Download citation
DOI: https://doi.org/10.1007/978-981-99-5847-4_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-5846-7
Online ISBN: 978-981-99-5847-4
eBook Packages: Computer ScienceComputer Science (R0)