Abstract
Recently, there has been an increasing interest in language-based interactions with technology. Driven by the success of intelligent personal assistants, the number of conversation-based interactions is growing in several domains. Nevertheless, the literature highlights a lack of models specifically studied to analyse communication blocks and the level of accessibility and acceptability by the user that can characterise Human-Agent dialogue. This paper aims to learn how much an agent can be accessible, how much the communication is understandable, and if it brings to a successful conclusion by extending two known models, the UTAUT2 and CEM. For both models, we defined new indicators to analyse communicability, acceptability degrees and accessibility level using WCAG guidelines. We designed two conversational agents to test our models and conducted preliminary tests. A first agent is used to assist students in following remote digital courses, and a second to help older people in their daily activities and to monitor the indexes of active life defined to control the trend of older people’s physical-cognitive state.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
https://www.whoteach.it/ Last access: 2023–06-08.
References
Boonstra, L.: Definitive Guide to Conversational AI with Dialogflow and Google Cloud. Apress (2021)
Venkatesh, V., Morris, M.G., Davis, G.B., Davis, F.D.: User acceptance of information technology: toward a unified view. MIS Q., 425–478 (2003)
Venkatesh, V., Davis, F.D.: A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag. Sci. 46(2), 186–204 (2000)
Warshaw, P.R.: A new model for predicting behavioral intentions: an alternative to Fishbein. J. Mark. Res. 17(2), 153–172 (1980)
Mattos, B.A.M., Prates, R.O.: An overview of the communicability evaluation method for collaborative systems. In: IADIS International Conference WWW/Internet 2011, Rio de Janeiro. Proceedings of WWW/Internet 2011, pp. 129–136 (2011)
De Souza, C.S., Leitão, C.F.: Semiotic Engineering Methods for Scientific Research in HCI. Morgan and Claypool Publishers, San Rafael (2009)
Prates, R.O., de Souza, C.S., Barbosa, S.D.J.: A method for evaluating the communicability of user interfaces. ACM Interact. 7(1), 31–38 (2000)
Okonkwo, C.W., Ade-Ibijola, A.: Chatbots applications in education: a systematic review. Comput. Educ. Artif. Intell. 2, 100033 (2021)
WCAG 3.0: https://www.w3.org/TR/wcag-3.0/ and WCAG 2.1: https://wcag.it/. Accessed 24 Apr 2023
Medeiros, R.P., Ramalho, G.L., Falcão, T.P.: A systematic literature review on teaching and learning introductory programming in higher education. IEEE Trans. Educ. 62(2), 77–90 (2018)
Smutny, P., Schreiberova, P.: Chatbots for learning: a review of educational chatbots for the Facebook Messenger. Comput. Educ. 151, 103862 (2020)
Alias, S., Sainin, M.S., Soo Fun, T., Daut, N.: Identification of conversational intent pattern using pattern-growth technique for academic chatbot. In: Chamchong, R., Wong, K.W. (eds.) MIWAI 2019. LNCS (LNAI), vol. 11909, pp. 263–270. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33709-4_24
Wu, E.H.K., Lin, C.H., Ou, Y.Y., Liu, C.Z., Wang, W.K., Chao, C.Y.: Advantages and constraints of a hybrid model K-12 E-learning assistant chatbot. IEEE Access 8, 77788–77801 (2020)
Murad, D.F., Irsan, M., Akhirianto, P.M., Fernando, E., Murad, S.A., Wijaya, M.H.: Learning support system using chatbot in “Kejar C Package” homeschooling program. In: 2019 International Conference on Information and Communications Technology (ICOIACT), pp. 32–37. IEEE, July 2019
Lam, C.S.N., Chan, L.K., See, C.Y.H.: Converse, connect and consolidate–the development of an artificial intelligence chatbot for health sciences education. In: Frontiers in Medical and Health Sciences Education Conference. Bau Institute of Medical and Health Sciences Education, Li Ka Shing Faculty of Medicine, The University of Hong Kong (2018)
El Hefny, W., El Bolock, A., Herbert, C., Abdennadher, S.: Applying the character-based chatbots generation framework in education and healthcare. In: Proceedings of the 9th International Conference on Human-Agent Interaction, pp. 121–129, November 2021
Sreelakshmi, A.S., Abhinaya, S.B., Nair, A., Nirmala, S.J.: A question answering and quiz generation chatbot for education. In: 2019 Grace Hopper Celebration India (GHCI), pp. 1–6. IEEE, November 2019
https://rasa.com/. Accessed 24 Apr 2023
https://it.legacy.reactjs.org/. Accessed 24 Apr 2023
https://cloud.google.com/dialogflow?hl=it. Accessed 24 Apr 2023
https://docs.flutter.dev/. Accessed 24 Apr 2023
Chaves, A.P., Gerosa, M.A.: How should my chatbot interact? A survey on social characteristics in human–chatbot interaction design. Int. J. Hum. Comput. Interact., 1–30 (2020)
Rapp, A., Curti, L., Boldi, A.: The human side of human-chatbot interaction: a systematic literature review of ten years of research on text-based chatbots. Int. J. Hum. Comput. Stud., 102630 (2021)
De Veer, A.J., Peeters, J.M., Brabers, A.E., Schellevis, F.G., Rademakers, J.J.J., Francke, A.L.: Determinants of the intention to use e-health by community dwelling older people. BMC Health Serv. Res. 15(1), 1–9 (2015)
Liu, C.F., Tsai, Y.C., Jang, F.L.: Patients’ acceptance towards a web-based personal health record system: an empirical study in Taiwan. Int. J. Environ. Res. Public Health 10(10), 5191–5208 (2013)
Kohnke, A., Cole, M.L., Bush, R.: Incorporating UTAUT predictors for understanding home care patients’ and clinician’s acceptance of healthcare telemedicine equipment. J. Technol. Manag. Innov. 9(2), 29–41 (2014)
Cimperman, M., Brenčič, M.M., Trkman, P.: Analysing older users’ home telehealth services acceptance behavior—applying an extended UTAUT model. Int. J. Med. Inform. 90, 22–31 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Valtolina, S., Matamoros, R.A., Epifania, F. (2023). Methods for Evaluating Conversational Agents’ Communicability, Acceptability and Accessibility Degree. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. Lecture Notes in Computer Science, vol 14143. Springer, Cham. https://doi.org/10.1007/978-3-031-42283-6_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-42283-6_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42282-9
Online ISBN: 978-3-031-42283-6
eBook Packages: Computer ScienceComputer Science (R0)