skip to main content
10.1145/3555228.3555230acmotherconferencesArticle/Chapter ViewAbstractPublication PagessbesConference Proceedingsconference-collections
research-article

UX Requirements Matters: Guidelines to Support Software Teams on the Writing of Acceptance Criteria

Published: 05 October 2022 Publication History

Abstract

User Stories (USs) are artifacts that define requirements in agile software development and that are usually complemented by Acceptance Criteria (AC) which provide details about what should be implemented. These details often attempt to integrate User Experience (UX) requirements to their descriptions. This paper aims to introduce a set of guidelines that supports software teams to write UX requirements during the elaboration of AC. The guidelines are separated into Interaction Design (ID) and Visual Elements (VE). We conducted a case study with 10 agile teams that develop mobile applications that used the guidelines during a sprint. After that, we carried out a qualitative analysis of the UX requirements described in the USs/AC to evaluate the usage of the guidelines. We analyzed the guidelines’ acceptance regarding usefulness and ease of use based on feedback from the teams. We analyzed 242 AC and identified that 99 AC had UX requirements related to ID and 87 to VE. We identified that only 7 developers, spread over four teams, reported negative feedback about the guidelines. We conclude that teams were helped by the guidelines that respectively address details about the user’s interaction with the product, as well as the definition of the most adequate visual elements to enable this interaction.

References

[1]
Jim Buchan, Muneera Bano, Didar Zowghi, Stephen MacDonell, and Amrita Shinde. 2017. Alignment of Stakeholder Expectations about User Involvement in Agile Software Development. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering. ACM, New York, NY, USA, 334–343. https://doi.org/10.1145/3084226.3084251
[2]
Joelma Choma, Luciana A. M. Zaina, and Daniela Beraldo. 2016. UserX Story: Incorporating UX Aspects into User Stories Elaboration. In Human-Computer Interaction. Theory, Design, Development and Practice, Masaaki Kurosu (Ed.). Springer International Publishing, Cham, 131–140. https://doi.org/10.1007/978-3-319-39510-4_13
[3]
Mike Cohn. 2004. User Stories Applied: For Agile Software Development. Addison Wesley Longman Publishing Co., Inc., USA.
[4]
Fred D. Davis. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13, 3 (1989), 319–340. https://doi.org/10.2307/249008
[5]
Joseph L. Fleiss. 1981. Statistical Methods for Rates and Proportions. Wiley, London.
[6]
Ron Garland. 1991. The mid-point on a rating scale: Is it desirable. Marketing bulletin 2, 1 (1991), 66–70. https://www.rangevoting.org/MB_V2_N3_Garland.pdf
[7]
Jesse James Garrett. 2011. The elements of user experience: user-centered design for the web and beyond, Second Edition. New Riders, Berkeley, CA, USA.
[8]
Graham R Gibbs. 2007. Analyzing Qualitative Data(1st ed.). SAGE Publications Ltd, London.
[9]
Laura Hokkanen and Kaisa Väänänen-Vainio-Mattila. 2015. UX Work in Startups: Current Practices and Future Needs. In Agile Processes in Software Engineering and Extreme Programming. Springer International Publishing, Cham, 81–92. https://doi.org/10.1007/978-3-319-18612-2_7
[10]
Robert Johns. 2005. One Size Doesn’t Fit All: Selecting Response Scales For Attitude Items. Journal of Elections, Public Opinion and Parties 15, 2 (2005), 237–264. https://doi.org/10.1080/13689880500178849
[11]
Fernando Kamei, Gustavo Pinto, Bruno Cartaxo, and Alexandre Vasconcelos. 2017. On the Benefits/Limitations of Agile Software Development: An Interview Study with Brazilian Companies. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering. ACM, New York, NY, USA, 1–6. https://doi.org/10.1145/3084226.3084278
[12]
Pariya Kashfi, Robert Feldt, and Agneta Nilsson. 2019. Integrating UX principles and practices into software development organizations: A case study of influencing events. Journal of Systems and Software 154 (2019), 37–58. https://doi.org/10.1016/j.jss.2019.03.066
[13]
Kati Kuusinen. 2015. Task allocation between UX specialists and developers in agile software development projects. In IFIP Conference on Human-Computer Interaction. Lecture Notes in Computer Science, Springer,., 27–44. https://doi.org/10.1007/978-3-319-22698-9_3
[14]
J. Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. Wiley, International Biometric Society 33 (1977), 159–174. https://doi.org/10.2307/2529310
[15]
Effie Lai-Chong Law, Virpi Roto, Marc Hassenzahl, Arnold P.O.S. Vermeeren, and Joke Kort. 2009. Understanding, scoping and defining user experience: a survey approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI ’09). ACM, New York, NY, USA, 719–728. https://doi.org/10.1145/1518701.1518813
[16]
Larissa A. Lopes, Eduardo G. Pinheiro, Tiago S. da Silva, and Luciana A. M. Zaina. 2018. Using UxD Artefacts to Support the Writing of User Stories: Findings of an Empirical Study with Agile Developers. In Proceedings of the 19th International Conference on Agile Software Development: Companion (Porto, Portugal) (XP ’18). ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3234152.3234158
[17]
Garm Lucassen, Fabiano Dalpiaz, Jan Martijn E.M. van der Werf, and Sjaak Brinkkemper. 2015. Forging high-quality User Stories: Towards a discipline for Agile Requirements. In IEEE 23rd International Requirements Engineering Conference (RE). Ottawa, Canada, 126–135. https://doi.org/10.1109/RE.2015.7320415
[18]
Garm Lucassen, Fabiano Dalpiaz, Jan Martijn E. M. van der Werf, and Sjaak Brinkkemper. 2016. The Use and Effectiveness of User Stories in Practice. In Requirements Engineering: Foundation for Software Quality, Maya Daneva and Oscar Pastor (Eds.). Springer International Publishing, Cham, 205–222. https://doi.org/10.1007/978-3-319-30282-9_14
[19]
Ana M. Moreno and Agustín Yagüe. 2012. Agile User Stories Enriched with Usability. In Agile Processes in Software Engineering and Extreme Programming, Claes Wohlin (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 168–176. https://doi.org/10.1007/978-3-642-30350-0_12
[20]
Jakob Nielsen. 2020. Usability Heuristics for User Interface Design. Nielsen Normamn Group (NNGroup). https://www.nngroup.com/articles/ten-usability-heuristics/ July 28, 2021.
[21]
Don Norman. 2018. Ad-Hoc Personas & Empathetic Focus. jnd.org. https://jnd.org/ad-hoc_personas_empathetic_focus/ July 28, 2021.
[22]
Dan North. 2006. Introducing behaviour driven development. Better Software Magazine(2006).
[23]
Anathan Pereira, Abner Cleto Filho, Eduardo Guerra, and Luciana A. M. Zaina. 2021. Towards a Pattern Language to Embed UX Information in Agile Software Requirements. In EuroPLoP’21: European Conference on Pattern Languages of Programs 2021, Graz, Austria, July 7 - 11, 2021. ACM, 18:1–18:8. https://doi.org/10.1145/3489449.3489991
[24]
Kai Petersen, Robert Feldt, Shahid Mujtaba, and Michael Mattsson. 2008. Systematic Mapping Studies in Software Engineering. In 12th International Conference on Evaluation and Assessment in Software Engineering (EASE). Bari, Italy, 68–77. https://doi.org/10.14236/ewic/EASE2008.8
[25]
Per Runeson and Martin Höst. 2009. Guidelines for conducting and reporting case study research in software engineering, In .Empirical software engineering 14, 2, 131–164. https://doi.org/10.1007/s10664-008-9102-8
[26]
Cristian Rusu, Silvana Roncagliolo, Virginica Rusu, and Cesar Collazos. 2011. A Methodology to Establish Usability Heuristics. In The 4th Int. Conf. on Advances in Computer-Human Interactions (ACHI 2011). Gosier, Guadeloupe, France, 59–62.
[27]
Eva-Maria Schön, Jörg Thomaschewski, and María José Escalona. 2017. Agile Requirements Engineering: A systematic literature review. Computer Standards & Interfaces 49 (2017), 79–91. https://doi.org/10.1016/j.csi.2016.08.011
[28]
Ken Schwaber and Jeff Sutherland. 2011. The scrum guide. Scrum Alliance 21, 19 (2011), 1. https://billlewistraining.com/wp-content/uploads/2017/02/PMP-Agile-Study-Materials.pdf
[29]
Tiago Silva da Silva, Milene Selbach Silveira, and Frank Maurer. 2013. Ten Lessons Learned from Integrating Interaction Design and Agile Development. In 2013 Agile Conference. Agile Conference 2013, Nashville, TN, USA, 42–49. https://doi.org/10.1109/AGILE.2013.11
[30]
Jonathan H. J. Souza. 2021. ACUX: um guia para escrita de aspectos de UX em Critérios de Aceitação de User Stories - In Portuguese. Master’s thesis. Universidade Federal de São Carlos, Brazil. https://repositorio.ufscar.br/handle/ufscar/14347
[31]
Yves Wautelet, Dries Gielis, Stephan Poelmans, and Samedi Heng. 2019. Evaluating the Impact of User Stories Quality on the Ability to Understand and Structure Requirements. In The Practice of Enterprise Modeling, Jaap Gordijn, Wided Guédria, and Henderik A. Proper (Eds.). Springer, Cham, 3–19. https://doi.org/10.1007/978-3-030-35151-9_1
[32]
Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in software engineering. In The Kluwer International Series in Software Engineering. Springer Science & Business Media.
[33]
Luciana AM Zaina, Helen Sharp, and Leonor Barroca. 2021. UX information in the daily work of an agile team: A distributed cognition analysis. International Journal of Human-Computer Studies 147 (2021), 102574. https://doi.org/10.1016/j.ijhcs.2020.102574

Cited By

View all
  • (2023)Unveiling Quality in Chatbot Conversations: Quantitative Analysis of Chatbot RequirementsProceedings of the XXII Brazilian Symposium on Software Quality10.1145/3629479.3629481(148-157)Online publication date: 7-Nov-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
SBES '22: Proceedings of the XXXVI Brazilian Symposium on Software Engineering
October 2022
457 pages
ISBN:9781450397353
DOI:10.1145/3555228
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 October 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Acceptance Criteria
  2. Mobile Application Development
  3. User Experience
  4. User Stories

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • São Paulo Research Foundation (FAPESP)
  • Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Conference

SBES 2022
SBES 2022: XXXVI Brazilian Symposium on Software Engineering
October 5 - 7, 2022
Virtual Event, Brazil

Acceptance Rates

Overall Acceptance Rate 147 of 427 submissions, 34%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)38
  • Downloads (Last 6 weeks)2
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Unveiling Quality in Chatbot Conversations: Quantitative Analysis of Chatbot RequirementsProceedings of the XXII Brazilian Symposium on Software Quality10.1145/3629479.3629481(148-157)Online publication date: 7-Nov-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media