Abstract
Intelligent dialog robot is now a primary necessity in online shopping, since it can significantly improve the service quality of the online merchants with fewer customer service staffs. And the core functionality of such robots is to identify the customers’ intention in order to provide better service. In this paper, we propose a domain-specific sentence encoder (DSSE) method for real-time online intent identification that hierarchically consists of a representation module and a classification module. We construct a large corpus using the dialog text from e-commerce platforms. We train the representation module of DSSE with the dialog corpus of all kinds of goods and then fine-tune the downstream multi-layer perceptron (MLP) classification module on the corpus of a specific kind of goods. Our model can be easily extended to new kinds of goods as it only needs a little training corpus to fine-tune the small MLP module on new goods chatting dialog text. In addition, our model has quick inference speed and relatively low resource requirement, as well as a higher intent identification accuracy score than BERT. Specifically, our model achieves 85.95% accuracy score and 81.62% F1 score. The inference time per-batch (size = 350) is 33 ms with CPU and only 5.8M parameters are loaded into the RAM.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmad, W., Bai, X., Peng, N., Chang, K.W.: Learning robust, transferable sentence representations for text classification, Septermber 2018
Che, J., Chen, H., Zeng, J., Zhang, L.-J.: A Chinese text correction and intention identification method for speech interactive context. In: Liu, S., Tekinerdogan, B., Aoyama, M., Zhang, L.-J. (eds.) EDGE 2018. LNCS, vol. 10973, pp. 127–134. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94340-4_10
Chen, X., Xu, L., Liu, Z., Sun, M., Luan, H.: Joint learning of character and word embeddings. In: Yang, Q., Wooldridge, M.J. (eds.) Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, 25–31 July 2015, pp. 1236–1242. AAAI Press (2015)
Cui, Y., Che, W., Liu, T., Qin, B., Wang, S., Hu, G.: Revisiting pre-trained models for chinese natural language processing. arXiv preprint arXiv:2004.13922 (2020)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, 2–7 June 2019, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics (2019)
Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24–26 April 2017, Conference Track Proceedings. OpenReview.net (2017)
Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. In: Gurevych, I., Miyao, Y. (eds.) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, 15–20 July 2018, Volume 1: Long Papers, pp. 328–339. Association for Computational Linguistics (2018)
Jiang, J., Wang, Z., Ying, H.: More information or less? Eidence from the biggest online shopping platform in china. In: Vogel, D., et al. (eds.) 24th Pacific Asia Conference on Information Systems, PACIS 2020, Dubai, UAE, 22–24 June 2020, p. 116 (2020)
Kai, X.: Study on feasibility and pattern about construction of university online shopping platform. In: The International Conference on E-Business and E-Government, ICEE 2010, 7–9 May 2010, Guangzhou, China, Proceedings, pp. 4577–4580. IEEE Computer Society (2010)
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: A lite BERT for self-supervised learning of language representations. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)
Lee, J.Y., Dernoncourt, F.: Sequential short-text classification with recurrent and convolutional neural networks. In: Knight, K., Nenkova, A., Rambow, O. (eds.) NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, 12–17 June 2016, pp. 515–520. The Association for Computational Linguistics (2016)
Lin, T., Xu, H.: Deep unknown intent detection with margin loss. In: Korhonen, A., Traum, D.R., Mà rquez, L. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, 28 July – 2 August 2019, Volume 1: Long Papers. pp. 5491–5496. Association for Computational Linguistics (2019)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Bengio, Y., LeCun, Y. (eds.) 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, 2–4 May 2013, Workshop Track Proceedings (2013)
Mishael, Q., Ayesh, A., Yevseyeva, I.: Users intention based on Twitter features using text analytics. In: Yin, H., Camacho, D., Tino, P., Tallón-Ballesteros, A.J., Menezes, R., Allmendinger, R. (eds.) IDEAL 2019, Part I. LNCS, vol. 11871, pp. 121–128. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33607-3_14
Ravuri, S.V., Stolcke, A.: Recurrent neural network and LSTM models for lexical utterance classification. In: INTERSPEECH 2015, 16th Annual Conference of the International Speech Communication Association, Dresden, Germany, 6–10 September 2015, pp. 135–139. ISCA (2015)
Sarikaya, R., Hinton, G.E., Ramabhadran, B.: Deep belief nets for natural language call-routing. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, 22–27 May 2011, Prague Congress Center, Prague, Czech Republic, pp. 5680–5683. IEEE (2011)
Shao, Z., Li, X., Guo, Y., Zhang, L.: Influence of service quality in sharing economy: understanding customers’ continuance intention of bicycle sharing. Electron. Commer. Res. Appl. 40, 100944 (2020)
Shu, L., Xu, H., Liu, B.: DOC: deep open classification of text documents. In: Palmer, M., Hwa, R., Riedel, S. (eds.) Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, 9–11 September 2017, pp. 2911–2916. Association for Computational Linguistics (2017)
Tür, G., Deng, L., Hakkani-Tür, D., He, X.: Towards deeper understanding: deep convex networks for semantic utterance classification. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2012, Kyoto, Japan, 25–30 March 2012. pp. 5045–5048. IEEE (2012)
Vaswani, A., et al.: Attention is all you need. In: Guyon, I., (eds.) 2017 Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, 4–9 December 2017, pp. 5998–6008. Long Beach, CA, USA (2017)
Yan, G., et al.: Unknown intent detection using Gaussian mixture model with an application to zero-shot intent classification. In: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J.R. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, 5–10 July 2020, pp. 1050–1060. Association for Computational Linguistics (2020)
Zhou, P., Qi, Z., Zheng, S., Xu, J., Bao, H., Xu, B.: Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling. In: Calzolari, N., Matsumoto, Y., Prasad, R. (eds.) COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, 11–16 December 2016, Osaka, Japan, pp. 3485–3495. ACL (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, C., Wang, Z., Yang, L., Liu, XY., Jiang, L. (2021). Domain-Specific Sentence Encoder for Intention Recognition in Large-Scale Shopping Platforms. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management . KSEM 2021. Lecture Notes in Computer Science(), vol 12816. Springer, Cham. https://doi.org/10.1007/978-3-030-82147-0_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-82147-0_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82146-3
Online ISBN: 978-3-030-82147-0
eBook Packages: Computer ScienceComputer Science (R0)