Abstract
Chatbots are conversational interfaces that enable human-like dialogue and can be designed in a textual chat format or a graphical interface with voice and embedding options. In the last year, there has been a significant growth in the emergence of chatbots in the market and this popularization has attracted the efforts of researchers to this area. Despite the existence of techniques to evaluate these tools, there is an urgent need to propose solutions that also support the chatbot design process. Currently, there is no knowledge of a specific list of requirements capable of supporting development teams in the process of designing these tools. In view of this, this directed study proposes a literature review aiming at deepening the knowledge about these tools and identifying important quality attributes in academic and industry sources. As a result, this directed study presents a list composed of 82 requirements related to Usefulness, Ease of Use and Presence to aid the design of these tools. These requirements presented in this study are useful to guide developers in the process of building quality chatbots, making this task less challenging and for researchers who aim to propose technologies that contribute to the development of better and better chatbots.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
de Andrade, G.G., Silva, G.R.S., Júnior, F.C.M.D., Santos, G.A., de Mendonça, F.L.L., de Sousa Júnior, R.T.: Evatalk: a chatbot system for the Brazilian Government Virtual School. In: Proceedings of the 22nd International Conference on Enterprise Information Systems (ICEIS 2020) (2020)
Borsci, S., et al.: The chatbot usability scale: the design and pilot of a usability scale for interaction with AI-based conversational agents. Personal Ubiquit. Comput. 26(1), 95–119 (2022)
Braun, D., Mendez, A.H., Matthes, F., Langen, M.: Evaluating natural language understanding services for conversational question answering systems. In: Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pp. 174–185 (2017)
Brill, T.M., Munoz, L., Miller, R.J.: Siri, Alexa, and other digital assistants: a study of customer satisfaction with artificial intelligence applications. J. Market. Manag. 35(15–16), 1401–1436 (2019)
ChatbotsLife. 9 tips to write the perfect chatbot script (2021). https://chatbotslife.com/9-tips-to-write-the-perfect-chatbot-script-bcb2695a5d0d
Correa, J., Viana, D., Teles, A.: Desenvolvendo chatbots com o dialogflow. Sociedade Brasileira de Computação (2021)
Garousi, Vahid, Felderer, Michael, Mäntylä, Mika V.., Rainer, Austen: Benefitting from the grey literature in software engineering research. In: Contemporary Empirical Methods in Software Engineering, pp. 385–413. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-32489-6_14
Gomes, B.R., Jacob Jr, A.F.L., Pinto, I.d.J.P., Colcher, S.: Ágata: um chatbot para difusão de práticas para educação ambiental. In: Anais Estendidos do XXVI Simpósio Brasileiro de Sistemas Multimídia e Web, pp. 85–89. SBC (2020)
Jain, M., Kumar, P., Kota, R., Patel, S.N.: Evaluating and informing the design of chatbots. In: Proceedings of the 2018 Designing Interactive Systems Conference, pp. 895–906 (2018)
Kim, M., Seo, B.G., Park, D.H.: Development process for user needs-based chatbot: focusing on design thinking methodology. J. Intell. Inf. Syst. 25(3), 221–238 (2019)
Kuligowska, K.: Commercial chatbot: performance evaluation, usability metrics and quality standards of embodied conversational agents. Profess. Center Bus. Res. 2 (2015)
Landbot. Chatbot checklist for new makers (2021). https://landbot.io/blog/chatbot-checklist. Accessed 05 Feb 2022
Langevin, R., Lordon, R.J., Avrahami, T., Cowan, B.R., Hirsch, T., Hsieh, G.: Heuristic evaluation of conversational agents. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–15 (2021)
Leite, J.C., Zhao, L., Kopczńska, S., Supakkul, S., Chung, L.: Report from the 6th international workshop on requirements patterns (REPA’16). ACM SIGSOFT Softw. Eng. Notes 42(1), 32–33 (2017)
Lokman, A.S., Ameedeen, M.A.: Modern chatbot systems: a technical review. In: Arai, K., Bhatia, R., Kapoor, S. (eds.) FTC 2018. AISC, vol. 881, pp. 1012–1023. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-02683-7_75
Magalhães, L.F.G.R.A.d.: Natureza morfológica dos chatbots. Ph.D. thesis, Universidade Católica Portuguesa (2018)
Maldonado, J.A.V., Cuadra, J.A.G.: Natural language interface to database using the dialogflow voice recognition and text conversion API. In: 2019 8th International Conference On Software Process Improvement (CIMPS), pp. 1–10. IEEE (2019)
Miner, A.S., Laranjo, L., Kocaballi, A.B.: Chatbots in the fight against the covid-19 pandemic. NPJ Dig. Med. 3(1), 1–4 (2020)
MobileTime. Panorama mobiletime - mapa do ecossistema brasileiro de bots 2021 (2021)
Nielsen, J.: Ten usability heuristics (2005)
Radziwill, N.M., Benton, M.C.: Evaluating quality of chatbots and intelligent conversational agents. arXiv preprint arXiv:1704.04579 (2017)
Ruggiano, N., Brown, E.L., Roberts, L., Suarez, C.V.F., Luo, Y., Hao, Z., Hristidis, V., et al.: Chatbots to support people with dementia and their caregivers: systematic review of functions and quality. J. Med. Internet Res. 23(6), e25006 (2021)
Sansonnet, J.-P., Leray, D., Martin, J.-C.: Architecture of a framework for generic assisting conversational agents. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 145–156. Springer, Heidelberg (2006). https://doi.org/10.1007/11821830_12
Sedoc, J., Ippolito, D., Kirubarajan, A., Thirani, J., Ungar, L., Callison-Burch, C.: Chateval: a tool for chatbot evaluation. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pp. 60–65 (2019)
Silva, K.K.C., Tierno, R.O., Branchine, S.M., Vilaça, D.S.S., Oliveira, F.H.M.: Desenvolvimento de ferramenta de chatbot como soluçao para a comunicaçao do ifb. In: Anais Estendidos do XVII Simpósio Brasileiro de Sistemas de Informação, pp. 185–188. SBC (2021)
Sugisaki, K., Bleiker, A.: Usability guidelines and evaluation criteria for conversational user interfaces: a heuristic and linguistic approach. In: Proceedings of the Conference on Mensch und Computer, pp. 309–319 (2020)
Take. Chatbot de qualidade: quais são as boas práticas para o contato inteligente (2020). https://www.take.net/blog/. Accessed 05 Feb 2022
Toader, D.C., et al.: The effect of social presence and chatbot errors on trust. Sustainability 12(1), 256 (2020)
Van Pinxteren, M.M., Pluymaekers, M., Lemmink, J.G.: Human-like communication in conversational agents: a literature review and research agenda. J. Serv. Manag. (2020)
Venkatesh, A., et al.: On evaluating and comparing conversational agents. arXiv preprint arXiv:1801.03625 vol. 4, pp. 60–68 (2018)
Weizenbaum, J.: Eliza-a computer program for the study of natural language communication between man and machine. Commun. ACM 9(1), 36–45 (1966)
Weng, F., Yang, R.J., Ho, H.J., Su, H.M.: A tam-based study of the attitude towards use intention of multimedia among school teachers. Appl. Syst. Innov. 1(3), 36 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mafra, M. et al. (2022). Defining Requirements for the Development of Useful and Usable Chatbots: An Analysis of Quality Attributes from Academy and Industry. In: Kurosu, M. (eds) Human-Computer Interaction. User Experience and Behavior. HCII 2022. Lecture Notes in Computer Science, vol 13304. Springer, Cham. https://doi.org/10.1007/978-3-031-05412-9_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-05412-9_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05411-2
Online ISBN: 978-3-031-05412-9
eBook Packages: Computer ScienceComputer Science (R0)