Abstract
Chatbot responses can be generated using traditional rule-based conversation design or through the use of large language models (LLMs). In this paper we compare the quality of responses provided by LLM-based chatbots with those provided by traditional conversation design. The results suggest that in some cases the use of LLMs could improve the quality of chatbot responses. The paper concludes by suggesting that a combination of approaches is the best way forward and suggests some directions for future work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The e-VITA project has received funding from the European Union H2020 Programme under grant agreement no. 101016453. The Japanese consortium received funding from the Japanese Ministry of Internal Affairs and Communication (MIC), Grant no. JPJ000595.
References
Cohen, M.H., Giangola, J.P., Balogh, J.: Voice User Interface Design. CA., USA, Addison-Wesley Professional, Redwood City (2004)
Pearl, C.: Designing Voice User Interfaces: Principles of Conversational Experiences. O’Reilly Media Inc, Sebastapol, USA (2016)
Deibel, D., Evanhoe, R.: Conversations with Things: UX Design for Chat and Voice. Rosenfeld Media, Brooklyn, New York, USA (2021)
Lizhou, F., Lingyao, L., Ma, Z., Lee, S., Yu, H., Hemphill, L: A Bibliometric Review of Large Language Models Research from 2017 to 2023. https://arxiv.org/abs/2304.02020. Accessed 3 Jun 2023 (2023)
e-VITA coach Homepage. https://www.e-vita.coach/. Accessed 3 Jun 2023
Greyling, C. Large language models are forcing conversational AI frameworks to look outward, https://cobusgreyling.medium.com/large-language-models-are-forcing-conversational-ai-frameworks-to-look-outward-54a1ad49ce63. Accessed 3 Jun 2023
Agbai, C. Chatbot design in the era of large language models (LLMs). https://azumo.com/insights/chatbot-design-in-the-era-of-large-language-models-llms. Accessed 3 Jun 2023
Kocaballi, A.B.: Conversational AI-powered design: ChatGPT as designer, user, and product. https://arxiv.org/abs/2302.07406. Accessed 3 Jun 2023 (2023)
Alto, V.: Modern Generative AI with ChatGPT and OpenAI models: leverage the capabilities of OpenAI’s LLM for productivity and innovation with GPT3 and GPT4. Packt Publishing, Birmingham (2023)
Moonash. ChatGPT vs other chatbot tools: a comprehensive comparison. https://everythingaboutchatgpt.com/chatgpt-comparison/#ChatGPT_vs_Rasa. Accessed 3 Jun 2023
Nichol, A.: Answering questions about structured data with Rasa Open Source and ChatGPT. https://rasa.com/blog/answering-questions-about-structured-data-with-rasa-open-source-and-chatgpt/. Accessed 3 Jun 2023
Ryan, M.: GPT-3 vs. Rasa chatbots. https://towardsdatascience.com/gpt-3-vs-rasa-chatbots-8b3041daf91d. Accessed 3 Jun 2023
Rasa Homepage. https://rasa.com/docs/rasa/. Accessed 3 Jun 2023
Marokkie, S.-V., McTear, M., Bi, Y.: A virtual companion for older adults using the Rasa Conversational AI framework. In: CONVERSATIONS 2022–the 6th International Workshop on Chatbot Research, Applications, and Design. https://conversations2022.files.wordpress.com/2022/11/conversations_2022_positionpaper_10_mctear.pdf. Accessed 3 Jun 2023 (2022)
ChatGPT Homepage. https://openai.com/blog/chatgpt. Accessed 3 Jun 2023
Thoppilan, R., De Freitas, D., Hall, D., et al.: LaMDA: Language Models for Dialog Applications. https://arxiv.org/abs/2201.08239. Accessed 3 Jun 2023
Bard FAQ. https://bard.google.com/faq. Accessed 3 Jun 2023
spacytextblob: A TextBlob sentiment analysis pipeline component for spaCy. https://spacytextblob.netlify.app. Accessed 3 Jun 2023
Wilcock, G., Jokinen, K.: Conversational AI and Knowledge Graphs for Social Robot Interaction. Late-Breaking Reports. In: The 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI-2022) (2022)
Nakhleh, A., Spitzer, S., Shehadeh, N.: ChatGPT’s response to the Diabetes Knowledge Questionnaire: Implications for diabetes education. Diabetes Technology & Therapeutics. https://doi.org/10.1089/dia.2023.0134 Accessed 3 Jun 2023 (2023)
So, P.: Voice Content and Usability. https://abookapart.com/products/voice-content-and-usability Accessed 3 Jun 2023 (2021)
ChatGPT Plugins. https://platform.openai.com/docs/plugins/introduction. Accessed 3 Jun 2023
Acknowledgements
Michael McTear received support from the e-VITA project (https://www.e-vita.coach/) (accessed on 23 April 2023). Sheen Varghese Marokkie and Yaxin Bi received support from the School of Computing, Ulster University (https://www.ulster.ac.uk/faculties/computing-engineering-and-the-built-environment/computing).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
McTear, M., Varghese Marokkie, S., Bi, Y. (2023). A Comparative Study of Chatbot Response Generation: Traditional Approaches Versus Large Language Models. In: Jin, Z., Jiang, Y., Buchmann, R.A., Bi, Y., Ghiran, AM., Ma, W. (eds) Knowledge Science, Engineering and Management. KSEM 2023. Lecture Notes in Computer Science(), vol 14118. Springer, Cham. https://doi.org/10.1007/978-3-031-40286-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-40286-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40285-2
Online ISBN: 978-3-031-40286-9
eBook Packages: Computer ScienceComputer Science (R0)