Skip to main content
SpringerLink
Log in

Understanding users’ requirements precisely: a double Bi-LSTM-CRF joint model for detecting user’s intentions and slot tags

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Understanding users’ requirements are essential to developing an effective AI service system, in which requirement expressions of users can be resolved into intent detection and slot filling tasks. In a lot of literature, the two tasks are normally considered as independent tasks and obtain satisfactory performance. Recently, many researchers have found that intent detection and slot filling can benefit each other since they always appear together in a sentence and may include shared information. Most of the existing joint models employ the structures of encoder and decoder and capture the cross-impact between two tasks by concatenation of hidden state information from two encoders, which ignore the dependencies among slot tags in specific intent. In this paper, we propose a novel Double-Bi-LSTM-CRF Model (DBLC), which can fit the dependency among hidden slot tags while considering the cross-impact between intent detection and slot filling. We also design and implement an intention chatbot on the tourism area, which can assist users to complete a travel plan through human-computer interaction. Extensive experiments show that our DBLC achieves state-of-the-art results on the benchmark ATIS, SNIPS, and multi-domain datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Dopierre T, Gravier C, Subercaze J, Logerais W (2020) Few-shot pseudo-labeling for intent detection. In: Proceedings of the 28th international conference on computational linguistics, pp 4993–5003

  2. Yan G, Fan, L, Li Q, Liu H, Zhang X, Wu X-M, Lam AY (2020) Unknown intent detection using gaussian mixture model with an application to zero-shot intent classification. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 1050–1060

  3. Liu J, Li Y, Lin M (2019) Review of intent detection methods in the human-machine dialogue system. J Phys Conf Ser 1267(1):012059

    Article  Google Scholar 

  4. Niu P, Chen Z, Song M et al (2019) A novel bi-directional interrelated model for joint intent detection and slot filling. arXiv preprint arXiv:1907.00390,

  5. Chelba C, Mahajan M, Acero A (2003) Speech utterance classification. In: 2003 IEEE international conference on acoustics, speech, and signal processing, 2003. Proceedings.(ICASSP’03)., vol 1. IEEE, pp I–I

  6. Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Mach Learn 39(2–3):135–168

    Article  Google Scholar 

  7. Hashemi H. B, Asiaee A, Kraft R (2016) Query intent detection using convolutional neural networks. In: International conference on web search and data mining, workshop on query understanding

  8. Bhargava A, Celikyilmaz A, Hakkani-Tür D, Sarikaya R (2013) Easy contextual intent prediction and slot detection. In: IEEE international conference on acoustics, speech and signal processing 2013. IEEE, pp 8337–8341

  9. Kapočiūtė-Dzikienė J (2020) Intent detection-based lithuanian chatbot created via automatic dnn hyper-parameter optimization. Front Artif Intell Appl 328:95–102

    Google Scholar 

  10. Ravuri S, Stolcke A (2015) Recurrent neural network and lstm models for lexical utterance classification. In: Sixteenth annual conference of the international speech communication association

  11. Lin Z, Feng M, Santos CNd, Yu M, Xiang B, Zhou B, Bengio Y (2017) A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130

  12. Xia C, Zhang C, Yan X, Chang Y, Yu PS (2018) Zero-shot user intent detection via capsule neural networks. arXiv preprint arXiv:1809.00385

  13. Tang H, Ji D, Zhou Q (2020) End-to-end masked graph-based crf for joint slot filling and intent detection. Neurocomputing 413:348–359

    Article  Google Scholar 

  14. Adel H, Schütze H (2019) Type-aware convolutional neural networks for slot filling. J Artif Intell Res 66:297–339

    Article  Google Scholar 

  15. Chen S, Yu S (2019) Wais: word attention for joint intent detection and slot filling. Proc AAAI Conf Artif Intell 33:9927–9928

    MathSciNet  Google Scholar 

  16. Ni P, Li Y, Li G, Chang V (2020) Natural language understanding approaches based on joint task of intent detection and slot filling for iot voice interaction. Neural Comput Appl 1–18

  17. Wang Y, Shen Y, Jin H(2018) A bi-model based rnn semantic frame parsing model for intent detection and slot filling. arXiv preprint arXiv:1812.10235

  18. Kim J-K, Tur G, Celikyilmaz A, Cao B, Wang Y-Y (2016) Intent detection using semantically enriched word embeddings. In: 2016 IEEE spoken language technology workshop (SLT). IEEE 2016, pp 414–419

  19. Mesnil G, Dauphin Y, Yao K, Bengio Y, Deng L, Hakkani-Tur D, He X, Heck L, Tur G, Yu D et al (2014) Using recurrent neural networks for slot filling in spoken language understanding. IEEE/ACM Trans Audio Speech Lang Process 23(3):530–539

    Article  Google Scholar 

  20. Guo D, Tur G, Yih W-T, Zweig G (2014) Joint semantic utterance classification and slot filling with recursive neural networks. In:2014 IEEE spoken language technology workshop (SLT). IEEE 2014:554–559

    Google Scholar 

  21. Liu B, Lane I (2015) Recurrent neural network structured output prediction for spoken language understanding. In: Proc. NIPS workshop on machine learning for spoken language understanding and interactions

  22. Xu P, Sarikaya R (2013) Convolutional neural network based triangular crf for joint intent detection and slot filling. In: 2013 IEEE workshop on automatic speech recognition and understanding. IEEE 2013, pp 78–83

  23. Kurata G, Xiang B, Zhou B, Yu M (2016) Leveraging sentence-level information with encoder lstm for semantic slot filling. arXiv preprint arXiv:1601.01530

  24. Deoras A, Sarikaya R (2013) Deep belief network based semantic taggers for spoken language understanding. In: Interspeech, pp 2713–2717

  25. Sukhbaatar S, Weston J, Fergus R et al (2015) End-to-end memory networks. Adv Neural Inf Process Syst 28:2440–2448

    Google Scholar 

  26. Zhang X, Wang H (2016) A joint model of intent determination and slot filling for spoken language understanding. IJCAI 16:2993–2999

    Google Scholar 

  27. Liu B, Lane I (2016) Joint online spoken language understanding and language modeling with recurrent neural networks. arXiv preprint arXiv:1609.01462

  28. Liu B, Lane I (2016) Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv preprint arXiv:1609.01454

  29. Goo C-W, Gao G, Hsu Y-K, Huo C-L, Chen T-C, Hsu K-W, Chen Y-N (2018) Slot-gated modeling for joint slot filling and intent prediction. In: Proceedings of the 2018 conference of the North American chapter of the Association for computational linguistics: human language technologies, vol 2 (Short Papers), pp 753–757

  30. Qin L, Che W, Li Y, Wen H, Liu T (2019) A stack-propagation framework with token-level intent detection for spoken language understanding. arXiv preprint arXiv:1909.02188

  31. Qin L, Ni M, Zhang Y, Che W (2020) Cosda-ml: multi-lingual code-switching data augmentation for zero-shot cross-lingual nlp. arXiv preprint arXiv:2006.06402

  32. Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (1)

  33. Sun Y, Wang S, Li Y, Feng S, Tian H, Wu H, Wang H (2020) Ernie 2.0: a continual pre-training framework for language understanding. In: Proceedings of the AAAI conference on artificial intelligence, vol 34(05), pp 8968–8975

  34. Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov RR, Le QV (2019) Xlnet: generalized autoregressive pretraining for language understanding. In: Advances in neural information processing systems, vol 32

  35. Chen Q, Zhuo Z, Wang W (2019) Bert for joint intent classification and slot filling. arXiv preprint arXiv:1902.10909

  36. Zhang Z, Zhang Z, Chen H, Zhang Z (2019) A joint learning framework with bert for spoken language understanding. IEEE Access 7:168 849-168 858

  37. Bansal H, Khan R (2018) A review paper on human computer interaction. Int J Adv Res Comput Sci Softw Eng 8:53–56

    Article  Google Scholar 

  38. Murtarelli G, Gregory A, Romenti S (2021) A conversation-based perspective for shaping ethical human-machine interactions: the particular challenge of chatbots. J Bus Res 129:927–935

    Article  Google Scholar 

  39. Papaioannou I, Dondrup C, Novikova J, Lemon O (2017) Hybrid chat and task dialogue for more engaging hri using reinforcement learning. In: (2017) 26th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE 2017:593–598

  40. Li K-C, Chang M, Wu K-H (2020) Developing a task-based dialogue system for English language learning. Educ Sci 10(11):306

    Article  Google Scholar 

  41. Adamopoulou E, Moussiades L (2020) An overview of chatbot technology. In: IFIP international conference on artificial intelligence applications and innovations. Springer, pp 373–383

  42. Adam M, Wessel M, Benlian A (2021) Ai-based chatbots in customer service and their effects on user compliance. Electron Mark 31:427–445

    Article  Google Scholar 

  43. Eleni A, Lefteris M (2020) Chatbots: History, technology, and applications. Mach Learn Appl 2:100006

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (No. 61902090, 61772159, 61832004), the Natural Science Foundation of Shandong Province (No. ZR2020KF019) and by State Key Laboratory of Communication Content Cognition, People’s Daily Online,(No. 2).

Author information

Authors and Affiliations

  1. State Key Laboratory of Communication Content Cognition, People’s Daily Online, Beijing, 100733, China

    Chunshan Li

  2. Department of Computer Science, Harbin Institute of Technology, Weihai, 264209, Shandong, China

    Chunshan Li, Yingli Zhou, Guoqing Chao & Dianhui Chu

Authors
  1. Chunshan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingli Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoqing Chao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dianhui Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Chunshan Li or Dianhui Chu.

Ethics declarations

Conflict of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this article. They have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, C., Zhou, Y., Chao, G. et al. Understanding users’ requirements precisely: a double Bi-LSTM-CRF joint model for detecting user’s intentions and slot tags. Neural Comput & Applic 34, 13639–13648 (2022). https://doi.org/10.1007/s00521-022-07171-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07171-y

Keywords

Search

Navigation