Geovana Ramos Sousa Silva
Edna Dias Canedo
ABSTRACT
As conversational assistants and natural language interfaces proliferate, the demand for a precise understanding of quality software requirements for chatbots becomes increasingly critical. In this work, we adopted a quantitative methodology, scrutinizing a dataset composed of conversational requirements from a diverse range of agile projects for chatbot development, and identified meaningful patterns in the language and structure utilized. Our investigation led to significant findings, revealing the importance of structured documentation, conversation flow, and user interaction in the development of chatbots, with the most desired quality attributes being capability, naturalness, straightforwardness, and clarity. In addition, a significant emphasis was placed on feature development and meeting acceptance criteria. The research also illuminated the iterative nature of chatbot development, with a recurrent presence of verbs related to improvement or refactoring. While less pronounced, the roles of documentation and testing in ensuring chatbot quality and effectiveness were also noted. This work provides valuable insights into chatbot requirements management and the significance of quality attributes in chatbot development.
- Eleni Adamopoulou and Lefteris Moussiades. 2020. Chatbots: History, technology, and applications. Machine Learning with Applications 2 (2020), 100006.Google Scholar
Cross Ref
- Daniel Bouskela, Alberto Falcone, Alfredo Garro, Audrey Jardin, Martin Otter, Nguyen Thuy, and Andrea Tundis. 2022. Formal requirements modeling for cyber-physical systems engineering: an integrated solution based on FORM-L and Modelica. Requirements Engineering 27, 1 (March 2022), 1–30. https://doi.org/10.1007/s00766-021-00359-zGoogle ScholarDigital Library
- Josip Božić. 2022. Ontology-based metamorphic testing for chatbots. Software Quality Journal 30, 1 (March 2022), 227–251. https://doi.org/10.1007/s11219-020-09544-9Google ScholarDigital Library
- Jordi Cabot, Loli Burgueño, Robert Clarisó, Gwendal Daniel, Jorge Perianez-Pascual, and Roberto Rodriguez-Echeverria. 2021. Testing challenges for NLP-intensive bots. In 2021 IEEE/ACM Third International Workshop on Bots in Software Engineering (BotSE). IEEE, 31–34. https://doi.org/10.1109/BotSE52550.2021.00014Google ScholarCross Ref
- Maxwell Crouse and Kenneth Forbus. 2020. Towards High-Precision Understanding of Comparative Analysis Problems Expressed in Natural Language. In Proceedings of the 33rd International Workshop on Qualitative Reasoning.Google Scholar
- Serena Di Gaetano and Pietro Diliberto. 2018. Chatbots and conversational interfaces: Three domains of use. In Fifth International Workshop on Cultures of Participation in the Digital Age, Vol. 2101. CEUR-WS.org, 62–70.Google Scholar
- António M. S. Ferreira., Alberto Rodrigues da Silva., and Ana C. R. Paiva.2022. Towards the Art of Writing Agile Requirements with User Stories, Acceptance Criteria, and Related Constructs. In Proceedings of the 17th International Conference on Evaluation of Novel Approaches to Software Engineering - ENASE. INSTICC, SciTePress, 477–484. https://doi.org/10.5220/0011082000003176Google ScholarCross Ref
- Asbjørn Følstad and Marita Skjuve. 2019. Chatbots for Customer Service: User Experience and Motivation. In Proceedings of the 1st International Conference on Conversational User Interfaces (Dublin, Ireland) (CUI ’19). ACM, New York, NY, USA, Article 1, 9 pages. https://doi.org/10.1145/3342775.3342784Google ScholarDigital Library
- Tom Gilb. 1981. System attribute specification: a cornerstone of software engineering. ACM SIGSOFT Software Engineering Notes 6, 3 (July 1981), 78–79. https://doi.org/10.1145/1010832.1010862Google ScholarDigital Library
- Seong-Soo Jeong and Yeong-Seok Seo. 2019. Improving response capability of chatbot using twitter. Journal of Ambient Intelligence and Humanized Computing1 (June 2019), 1–14. https://doi.org/10.1007/s12652-019-01347-6Google ScholarCross Ref
- Nurul Muizzah Johari, Puteri N. E. Nohuddin, Ahmad Hanif Ahmad Baharin, Noor Azuddin Yakob, and Mohammad Javad Ebadi. 2022. Features requirement elicitation process for designing a chatbot application. IET Networks (2022). https://doi.org/10.1049/ntw2.12071Google ScholarCross Ref
- Nurul Muizzah Johari, Halimah Badioze Zaman, and Puteri N. E. Nohuddin. 2019. Ascertain Quality Attributes for Design and Development of New Improved Chatbots to Assess Customer Satisfaction Index (CSI): A Preliminary Study. In Advances in Visual Informatics. Springer, Cham, Switzerland, 135–146. https://doi.org/10.1007/978-3-030-34032-2_13Google ScholarDigital Library
- Manoj Joshi and Rakesh Kumar Sharma. 2020. An Analytical Study and Review of open Source Chatbot framework, RASA. International Journal of Engineering Research & Technology 9, 6 (June 2020), 1011–1014. https://doi.org/10.17577/IJERTV9IS060723Google ScholarCross Ref
- Diego Kovags, Fábio Luíz Falchi, and André Ronaldo Rivas. 2019. Analysis of the Utilization of Scrum Framework Effort Estimation Metrics in Federal Public Administration. In Proceedings of the XVIII Brazilian Symposium on Software Quality(SBQS ’19). ACM, New York, NY, USA, 30–38. https://doi.org/10.1145/3364641.3364645Google ScholarDigital Library
- Achaporn Kwangsawad and Aungkana Jattamart. 2022. Overcoming customer innovation resistance to the sustainable adoption of chatbot services: A community-enterprise perspective in Thailand. Journal of Innovation & Knowledge 7, 3 (2022), 100211.Google ScholarCross Ref
- Malu Mafra, Kennedy Nunes, Adailton Castro, Adriana Lopes, Ana Carolina Oran, Geraldo Braz Junior, João Almeida, Anselmo Paiva, Aristofanes Silva, Simara Rocha, 2022. Defining Requirements for the Development of Useful and Usable Chatbots: An Analysis of Quality Attributes from Academy and Industry. In International Conference on Human-Computer Interaction. Springer, 479–493.Google ScholarDigital Library
- Michał Marcińczuk. 2017. Lemmatization of Multi-word Common Noun Phrases and Named Entities in Polish. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017. INCOMA Ltd., Varna, Bulgaria, 483–491. https://doi.org/10.26615/978-954-452-049-6_064Google ScholarCross Ref
- Suéllen Martinelli, Nicolas Nascimento, Jonathan Souza, Afonso Sales, and Luciana Zaina. 2022. UX Requirements Matters: Guidelines to Support Software Teams on the Writing of Acceptance Criteria. In SBES ’22: Proceedings of the XXXVI Brazilian Symposium on Software Engineering. ACM, New York, NY, USA, 398–408. https://doi.org/10.1145/3555228.3555230Google ScholarDigital Library
- Geovana R. S. Silva. and Edna Dias Canedo.2022. Requirements Engineering Challenges and Techniques in Building Chatbots. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART. INSTICC, SciTePress, 180–187. https://doi.org/10.5220/0010801800003116Google ScholarCross Ref
- Geovana Ramos Sousa Silva and Edna Dias Canedo. 2022. Towards User-Centric Guidelines for Chatbot Conversational Design. International Journal of Human–Computer Interaction (Sept. 2022), 1–23. https://doi.org/10.1080/10447318.2022.2118244Google ScholarCross Ref
- Gabriel Soares, Sávio Freire, Nicolli Rios, Boris Pérez, Camilo Castellanos, Darío Correal, Manoel Mendonça, Clemente Izurieta, Carolyn Seaman, and Rodrigo Spínola. 2023. Investigating How Agile Software Practitioners Repay Technical Debt in Software Projects. In Proceedings of the XXI Brazilian Symposium on Software Quality(SBQS ’22). ACM, New York, NY, USA, Article 13, 10 pages. https://doi.org/10.1145/3571473.3571499Google ScholarDigital Library
- Renata Souza, Karla Malta, Roselane Silva, Paulo Masiero, Eduardo Almeida, and Ivan Machado. 2019. A Case Study about Startups’ Software Development Practices: A Preliminary Result. In Proceedings of the XVIII Brazilian Symposium on Software Quality(SBQS ’19). ACM, New York, NY, USA, 198–203. https://doi.org/10.1145/3364641.3364663Google ScholarDigital Library
- spaCy. 2023. Portuguese spaCy Models Documentation — Model pt_core_news_sm - version 3.6.0. https://spacy.io/models/pt [Online; accessed 18. Jul. 2023].Google Scholar
- Diana-Cezara Toader, Grațiela Boca, Rita Toader, Mara Măcelaru, Cezar Toader, Diana Ighian, and Adrian T Rădulescu. 2019. The effect of social presence and chatbot errors on trust. Sustainability 12, 1 (2019), 256.Google ScholarCross Ref
- Yuli Vasiliev. 2020. Natural language processing with Python and spaCy: A practical introduction. No Starch Press.Google Scholar
- Ying Xu, Jianyu Zhang, and Guangkuan Deng. 2022. Enhancing customer satisfaction with chatbots: The influence of communication styles and consumer attachment anxiety. Frontiers in Psychology 13 (2022), 17 pages. https://doi.org/10.3389/fpsyg.2022.902782Google ScholarCross Ref
Index Terms
- Unveiling Quality in Chatbot Conversations: Quantitative Analysis of Chatbot Requirements
Recommendations
Action-a-Bot: Exploring Human-Chatbot Conversations for Actionable Instruction Giving and Following
CSCW'22 Companion: Companion Publication of the 2022 Conference on Computer Supported Cooperative Work and Social ComputingConversation serves as one critical mechanism for knowledge-sharing and instruction-giving in collaborative work. Conversation allows people to take turns to make contributions, plan joint actions, align shared understanding of work status and resolve ...
Chatbot with Touch and Graphics: An Interaction of Users for Emotional Expression and Turn-taking
CUI '20: Proceedings of the 2nd Conference on Conversational User InterfacesUse of chatbots for emotional exchange is recently increasing in various domains. However, as existing chatbots have been considered in terms of natural language processing techniques for interaction with text-based chatting, chatbot interaction with ...
Chatbot Theory: A Naïve and Elementary Theory for Dialogue Management
Intelligent Data Engineering and Automated Learning – IDEAL 2018AbstractDue to the increasing interested and use of chatbot, its properties and operation possibilities shall be proper realized matching both safety and security issues as well as present the several uses and compositions that this technology supports. ...
Comments