Abstract
Advances in dialogue systems have recently been made in various fields as an easy to use and inexpensive option to support or replace workers. However, developing dialogue systems that produce satisfactory responses to user queries on par with human workers still presents significant challenges. The primary purpose of this review is to analyse prominent studies on dialogue systems in the literature. Comparison frameworks were developed to perform an in-depth analysis in terms of approaches, data sets and evaluation metrics. Unlike previous reviews, we thoroughly examined how reinforcement learning is applied to dialogue systems. We also analysed studies attempting to interleave the two main types of dialogue systems (i.e. open-domain dialogue and task-oriented dialogue). We present some open-source platforms for developing dialogue systems. Finally, we identified research gaps and discussed potential research directions.
Similar content being viewed by others
Data availability
Data sharing not applicable to this article as no data sets were generated or analysed during the current study.
Change history
07 February 2024
A Correction to this paper has been published: https://doi.org/10.1007/s00521-024-09416-4
References
Zhang Z, Takanobu R, Zhu Q, Huang M, Zhu X (2020) Recent advances and challenges in task-oriented dialog systems. Sci China Technol Sci 63(10):2011–2027
Aleedy M, Shaiba H, Bezbradica M (2019) Generating and analyzing chatbot responses using natural language processing. Int J Adv Comput Sci Appl 10(9):60–68
Li Y, Su H, Shen X, Li W, Cao Z, Niu S (2017) Dailydialog: a manually labelled multi-turn dialogue dataset. In: Proceedings of The 8th international joint conference on natural language processing (IJCNLP 2017)
Zang X, Rastogi A, Sunkara S, Gupta R, Zhang J, Chen J (2020) Multiwoz 2.2: a dialogue dataset with additional annotation corrections and state tracking baselines. In: Proceedings of the 2nd workshop on natural language processing for conversational AI, ACL 2020, pp 109–117
Sun K, Moon S, Crook PA, Roller S, Silvert B, Liu, B, Wang Z, Liu H, Cho E, Cardie C (2021) Adding chit-chat to enhance task-oriented dialogues. In: Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 1570–1583
Zhao T, Lu A, Lee K, Eskenazi M (2017) Generative encoder–decoder models for task-oriented spoken dialog systems with chatting capability. In: Proceedings of the 18th annual SIGdial meeting on discourse and dialogue, pp 27–36
Adamopoulou E, Moussiades L (2020) Chatbots: history, technology, and applications. Mach Learn Appl 2:100006
Suhaili SM, Salim N, Jambli MN (2021) Service chatbots: a systematic review. Expert Syst Appl 184:115461
Chen H, Liu X, Yin D, Tang J (2017) A survey on dialogue systems: recent advances and new frontiers. ACM SIGKDD Explor Newsl 19(2):25–35
Stolcke A, Ries K, Coccaro N, Shriberg E, Bates R, Jurafsky D, Taylor P, Martin R, Ess-Dykema CV, Meteer M (2000) Dialogue act modeling for automatic tagging and recognition of conversational speech. Comput Linguist 26(3):339–373
Louvan S, Magnini B (2020) Recent neural methods on slot filling and intent classification for task-oriented dialogue systems: a survey. In: Proceedings of the 28th international conference on computational linguistics, pp 480–496
Liu B, Lane I (2016) Attention-based recurrent neural network models for joint intent detection and slot filling. Interspeech 2016:685–689
Ramadan O, Budzianowski P, Gasic M (2018) Large-scale multi-domain belief tracking with knowledge sharing. In: Proceedings of the 56th annual meeting of the association for computational linguistics (volume 2: short papers), pp 432–437
Balaraman V, Sheikhalishahi S, Magnini B (2021) Recent neural methods on dialogue state tracking for task-oriented dialogue systems: a survey. In: Proceedings of the 22nd annual meeting of the special interest group on discourse and dialogue. Association for Computational Linguistics, Singapore and Online, pp 239–251. https://aclanthology.org/2021.sigdial-1.25
Ni J, Young T, Pandelea V, Xue F, Cambria E (2022) Recent advances in deep learning based dialogue systems: a systematic survey. Artif Intell Rev. https://doi.org/10.1007/s10462-022-10248-8
Huang M, Zhu X, Gao J (2020) Challenges in building intelligent open-domain dialog systems. ACM Trans Inf Syst 38(3):1–32
Hussain S, Ameri Sianaki O, Ababneh N (2019) A survey on conversational agents/chatbots classification and design techniques. In: Workshops of the international conference on advanced information networking and applications. Springer, pp 946–956
Weld H, Huang X, Long S, Poon J, Han SC (2022) A survey of joint intent detection and slot filling models in natural language understanding. ACM Comput Surv. https://doi.org/10.1145/3547138
Agarwal R, Wadhwa M (2020) Review of state-of-the-art design techniques for chatbots. SN Comput Sci 1(5):1–12
Liu B, Lane I (2017) Iterative policy learning in end-to-end trainable task-oriented neural dialog models. In: 2017 IEEE automatic speech recognition and understanding workshop (ASRU). IEEE, pp 482–489
Takanobu R, Liang R, Huang M (2020) Multi-agent task-oriented dialog policy learning with role-aware reward decomposition. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 625–638
Zhang Z, Liao L, Zhu X, Chua T-S, Liu Z, Huang Y, Huang M (2020) Learning goal-oriented dialogue policy with opposite agent awareness. In: Proceedings of the 1st conference of the Asia-Pacific chapter of the association for computational linguistics and the 10th international joint conference on natural language processing, pp 122–132
Tseng B-H, Dai Y, Kreyssig F, Byrne B (2021) Transferable dialogue systems and user simulators. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers). Association for Computational Linguistics, pp 152–166. https://aclanthology.org/2021.acl-long.13
Papangelis A, Wang Y-C, Molino P, Tür G (2019) Collaborative multi-agent dialogue model training via reinforcement learning. In: Proceedings of the 20th annual SIGdial meeting on discourse and dialogue, pp 92–102
Saha T, Saha S, Bhattacharyya P (2020) Towards sentiment aided dialogue policy learning for multi-intent conversations using hierarchical reinforcement learning. PLoS ONE 15(7):0235367
Saha T, Saha S, Bhattacharyya P (2022) Towards sentiment-aware multi-modal dialogue policy learning. Cogn Comput 1–15
Ultes S, Maier W (2021) User satisfaction reward estimation across domains: domain-independent dialogue policy learning. Dialogue Discourse 12(2):81–114
Zhang R, Wang Z, Zheng M, Zhao Y, Huang Z (2021) Emotion-sensitive deep dyna-q learning for task-completion dialogue policy learning. Neurocomputing 459:122–130
Geishauser C, Hu S, Lin H, Lubis N, Heck M, Feng S, van Niekerk C, Gašić M (2021) What does the user want? Information gain for hierarchical dialogue policy optimisation. In: 2021 IEEE automatic speech recognition and understanding workshop (ASRU), pp 969–976. https://doi.org/10.1109/ASRU51503.2021.9687856
Peng B, Li X, Gao J, Liu J, Chen Y-N, Wong K-F (2018) Adversarial advantage actor-critic model for task-completion dialogue policy learning. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 6149–6153
Takanobu R, Zhu H, Huang M (2019) Guided dialog policy learning: Reward estimation for multi-domain task-oriented dialog. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 100–110
Liu B, Lane I (2018) Adversarial learning of task-oriented neural dialog models. In: Proceedings of the 19th annual SIGdial meeting on discourse and dialogue, pp 350–359
Lipton Z, Li X, Gao J, Li L, Ahmed F, Deng L (2018) Bbq-networks: efficient exploration in deep reinforcement learning for task-oriented dialogue systems. In: Proceedings of the AAAI conference on artificial intelligence, vol 32
Gordon-Hall G, Gorinski PJ, Lampouras G, Iacobacci I (2020) Show us the way: learning to manage dialog from demonstrations. CoRR arxiv:2004.08114
Liu B, Tür G, Hakkani-Tur D, Shah P, Heck L (2018) Dialogue learning with human teaching and feedback in end-to-end trainable task-oriented dialogue systems. In: Proceedings of the 2018 conference of the north American chapter of the association for computational linguistics: human language technologies, volume 1 (long papers), pp 2060–2069
Hosseini-Asl E, McCann B, Wu C-S, Yavuz S, Socher R (2020) A simple language model for task-oriented dialogue. Adv Neural Inf Process Syst 33:20179–20191
Wen T-H, Vandyke D, Mrkšić N, Gasic M, Barahona LMR, Su P-H, Ultes S, Young S (2017) A network-based end-to-end trainable task-oriented dialogue system. In: Proceedings of the 15th conference of the European chapter of the association for computational linguistics: volume 1, long papers, pp 438–449
Liu B, Lane I (2017) An end-to-end trainable neural network model with belief tracking for task-oriented dialog. Proc Interspeech 2017:2506–2510
Budzianowski P, Vulic I (2019) Hello, it’s gpt-2-how can i help you? Towards the use of pretrained language models for task-oriented dialogue systems. EMNLP-IJCNLP 2019:15
Peng B, Li C, Li J, Shayandeh S, Liden L, Gao J (2021) Soloist: buildingtask bots at scale with transfer learning and machine teaching. Trans Assoc Comput Linguist 9:807–824
Lin Z, Madotto A, Winata GI, Fung P (2020) Mintl: Minimalist transfer learning for task-oriented dialogue systems. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp 3391–3405
Kulhánek J, Hudeček V, Nekvinda T, Dušek O (2021) AuGPT: Auxiliary tasks and data augmentation for end-to-end dialogue with pre-trained language models. In: Proceedings of the 3rd workshop on natural language processing for conversational AI, pp 198–210. Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.nlp4convai-1.19. https://aclanthology.org/2021.nlp4convai-1.19
Ham D, Lee J-G, Jang Y, Kim K-E (2020) End-to-end neural pipeline for goal-oriented dialogue systems using gpt-2. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 583–592
Xing C, Wu Y, Wu W, Huang Y, Zhou M (2018) Hierarchical recurrent attention network for response generation. In: Proceedings of the AAAI conference on artificial intelligence, vol 32
Ghazvininejad M, Brockett C, Chang M-W, Dolan B, Gao J, Yih W-t, Galley M (2018) A knowledge-grounded neural conversation model. In: Proceedings of the AAAI conference on Artificial Intelligence, vol 32
Bao S, He H, Wang F, Lian R, Wu H (2019) Know more about each other: evolving dialogue strategy via compound assessment. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 5382–5391
Ko W-J, Ray A, Shen Y, Jin H (2020) Generating dialogue responses from a semantic latent space. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp 4339–4349
Zhang S, Dinan E, Urbanek J, Szlam A Kiela D, Weston J (2018) Personalizing dialogue agents: I have a dog, do you have pets too? In: ACL (1)
Zhou H, Huang M, Zhang T, Zhu X, Liu B (2018) Emotional chatting machine: Emotional conversation generation with internal and external memory. In: Proceedings of the AAAI conference on artificial intelligence, vol 32
Wang Y-H, Hsu J-H, Wu C-H, Yang T-H (2021) Transformer-based empathetic response generation using dialogue situation and advanced-level definition of empathy. In: 2021 12th international symposium on Chinese spoken language processing (ISCSLP). IEEE, pp 1–5
Choudhary R, Kawahara D (2022) Grounding in social media: An approach to building a chit-chat dialogue model. In: Proceedings of the 2022 conference of the North American chapter of the association for computational linguistics: human language technologies: student research workshop, pp 9–15. Association for computational linguistics, hybrid: Seattle, Washington + Online. https://doi.org/10.18653/v1/2022.naacl-srw.2. https://aclanthology.org/2022.naacl-srw.2
Kasahara T, Kawahara D, Tung N, Li S, Shinzato K, Sato T (2022) Building a personalized dialogue system with prompt-tuning. In: Proceedings of the 2022 conference of the North American chapter of the association for computational linguistics: human language technologies: student research workshop, pp 96–105
Yu Z, Black AW, Rudnicky AI (2017) Learning conversational systems that interleave task and non-task content. In: Proceedings of the 26th international joint conference on artificial intelligence, pp 4214–4220
Young T, Xing F, Pandelea V, Ni J, Cambria E (2022) Fusing task-oriented and open-domain dialogues in conversational agents. In: Association for the advancement of artificial intelligence (www.aaai.org)
Gür I, Hakkani-Tür D, Tür G, Shah P (2018) User modeling for task oriented dialogues. In: 2018 IEEE Spoken language technology workshop (SLT). IEEE, pp 900–906
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30
Stasaski K, Yang GH, Hearst MA (2020) More diverse dialogue datasets via diversity-informed data collection. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 4958–4968
Rashkin H, Smith EM, Li M, Boureau Y-L (2019) Towards empathetic open-domain conversation models: a new benchmark and dataset. In: ACL (1)
Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901
Thoppilan R, De Freitas D, Hall J, Shazeer N, Kulshreshtha A, Cheng, H-T, Jin A, Bos T, Baker L, Du Y, et al. (2022) Lamda: language models for dialog applications. arXiv preprint arXiv:2201.08239
Anil R, Dai AM, Firat O, Johnson M, Lepikhin D, Passos A, Shakeri S, Taropa E Bailey P, Chen Z, et al (2023) Palm 2 technical report. arXiv preprint arXiv:2305.10403
Zhou C, Li Q, Li C, Yu J, Liu Y, Wang G, Zhang K, Ji C, Yan Q, He L, et al (2023) A comprehensive survey on pretrained foundation models: a history from bert to chatgpt. arXiv preprint arXiv:2302.09419
OpenAI: Introducing ChatGPT—openai.com. https://openai.com/blog/chatgpt [Accessed 29-09-2023] (2022)
Hudeček V, Dušek O (2023) Are llms all you need for task-oriented dialogue? arXiv preprint arXiv:2304.06556
Zhang X, Peng B, Li K, Zhou J, Meng H (2023) Sgp-tod: building task bots effortlessly via schema-guided llm prompting. arXiv preprint arXiv:2305.09067
Zhao W, Zhao Y, Lu X, Wang S, Tong Y, Qin B (2023) Is chatgpt equipped with emotional dialogue capabilities? arXiv preprint arXiv:2304.09582
Fu Y, Inoue K, Chu C, Kawahara T (2023) Reasoning before responding: integrating commonsense-based causality explanation for empathetic response generation. arXiv preprint arXiv:2308.00085
Budzianowski P, Wen T-H, Tseng B-H, Casanueva I, Stefan U, Osman R, Gašić M (2018) Multiwoz—a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In: Proceedings of the 2018 conference on empirical methods in natural language processing (EMNLP)
Eric M, Goel R, Paul S, Sethi A, Agarwal S, Gao S, Kumar A, Goyal A, Ku P, Hakkani-Tur D (2020) MultiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking baselines. In: Proceedings of the twelfth language resources and evaluation conference, pp 422–428. European Language Resources Association, Marseille, France. https://aclanthology.org/2020.lrec-1.53
Henderson M, Thomson B, Williams JD (2014) The second dialog state tracking challenge. In: Proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (SIGDIAL), pp 263–272
Li X, Chen Y-N, Li L, Gao J, Celikyilmaz A (2017) End-to-end task-completion neural dialogue systems. In: Proceedings of the eighth international joint conference on natural language processing (volume 1: long papers), pp 733–743
Ultes S, Barahona LMR, Su P-H, Vandyke D, Kim D, Casanueva I, Budzianowski P, Mrkšić N, Wen T-H, Gasic M, et al (2017) Pydial: a multi-domain statistical dialogue system toolkit. In: Proceedings of ACL 2017, system demonstrations, pp 73–78
El Asri L, Schulz H, Sharma S, Zumer J, Harris J, Fine E, Mehrotra R, Suleman K (2017) Frames: a corpus for adding memory to goal-oriented dialogue systems. In: Proceedings of the 18th annual SIGdial meeting on discourse and dialogue, pp 207–219. Association for Computational Linguistics, Saarbrücken, Germany. https://doi.org/10.18653/v1/W17-5526. https://aclanthology.org/W17-5526
Chiu S, Li M, Lin Y-T, Chen Y-N (2022) Salesbot: transitioning from chit-chat to task-oriented dialogues. In: Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers), pp 6143–6158
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, London
Wang H, Peng B, Wong K-F (2020) Learning efficient dialogue policy from demonstrations through shaping. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 6355–6365
Gao J, Galley M, Li L et al (2019) Neural approaches to conversational ai. Foundations and trends® in information retrieval 13(2–3):127–298
Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34(6):26–38
Peng B, Li X, Gao J, Liu J, Wong K-F (2018) Deep Dyna-Q: integrating planning for task-completion dialogue policy learning. In: Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pp 2182–2192. Association for Computational Linguistics, Melbourne, Australia. https://doi.org/10.18653/v1/P18-1203. https://aclanthology.org/P18-1203
Brabra H, Báez M, Benatallah B, Gaaloul W, Bouguelia S, Zamanirad S (2021) Dialogue management in conversational systems: a review of approaches, challenges, and opportunities. IEEE Trans Cogn Dev Syst
Casanueva I, Budzianowski P, Su P-H, Ultes S, Rojas-Barahona L, Tseng B-H, Gašic M (2018) Feudal reinforcement learning for dialogue management in large domains. In: Proceedings of NAACL-HLT, pp 714–719
Li Z, Lee S, Peng B, Li J, Kiseleva J, de Rijke M, Shayandeh S, Gao J (2020) Guided dialogue policy learning without adversarial learning in the loop. Find Assoc Comput Linguist EMNLP 2020:2308–2317
Wu G, Fang W, Wang J, Cao J, Bao W, Ping Y, Zhu X, Wang Z (2021) Gaussian process based deep dyna-q approach for dialogue policy learning. Find Assoc Comput Linguist ACL-IJCNLP 2021:1786–1795
Peng B, Li X, Li L, Gao J, Celikyilmaz A, Lee S, Wong K-F (2017) Composite task-completion dialogue policy learning via hierarchical deep reinforcement learning. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 2231–2240. Association for Computational Linguistics, Copenhagen, Denmark. https://doi.org/10.18653/v1/D17-1237. https://aclanthology.org/D17-1237
Dai Y, Yu H, Jiang Y, Tang C, Li Y, Sun J (2020) A survey on dialog management: recent advances and challenges. arXiv preprint arXiv:2005.02233
Zhu Q, Zhang Z, Fang Y, Li X, Takanobu R, Li J, Peng B, Gao J, Zhu X, Huang M (2020) Convlab-2: An open-source toolkit for building, evaluating, and diagnosing dialogue systems. In: Proceedings of the 58th annual meeting of the association for computational linguistics: system demonstrations, pp 142–149
Miller A, Feng W, Batra D, Bordes A, Fisch A, Lu J, Parikh D, Weston J (2017) Parlai: a dialog research software platform. In: Proceedings of the 2017 conference on empirical methods in natural language processing: system demonstrations, pp 79–84
Bocklisch T, Faulkner J, Pawlowski N, Nichol A (2017) Rasa: open source language understanding and dialogue management. arXiv preprint arXiv:1712.05181
Zhao YJ, Li YL, Lin M (2019) A review of the research on dialogue management of task-oriented systems. J Phys Conf Ser 1267:012025
Liu C-W, Lowe R, Serban I, Noseworthy M, Charlin L, Pineau J (2016) How NOT to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 2122–2132. Association for Computational Linguistics, Austin, Texas. https://doi.org/10.18653/v1/D16-1230. https://aclanthology.org/D16-1230
Deriu J, Rodrigo A, Otegi A, Echegoyen G, Rosset S, Agirre E, Cieliebak M (2021) Survey on evaluation methods for dialogue systems. Artif Intell Rev 54(1):755–810
Takanobu R, Zhu Q, Li J, Peng B, Gao J, Huang M (2020) Is your goal-oriented dialog model performing really well? Empirical analysis of system-wise evaluation. In: Proceedings of the 21th annual meeting of the special interest group on discourse and dialogue, pp 297–310
Acknowledgements
The authors wish to acknowledge King Fahd University of Petroleum and Minerals (KFUPM) for providing the facilities to carry out this research. Many thanks are due to the anonymous referees for their detailed and helpful comments.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original online version of this article was revised to correct the affiliation of Author Moataz Ahmed.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Algherairy, A., Ahmed, M. A review of dialogue systems: current trends and future directions. Neural Comput & Applic 36, 6325–6351 (2024). https://doi.org/10.1007/s00521-023-09322-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-023-09322-1