Skip to main content
SpringerLink
Log in

An Empirical Study of User Story Quality and Its Impact on Open Source Project Performance

  • Conference paper
  • First Online:
Software Quality: Future Perspectives on Software Engineering Quality (SWQD 2021)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 404))

Included in the following conference series:

Abstract

When software development teams apply Agile Software Development practices, they commonly express their requirements as User Stories. We aim to study the quality of User Stories and its evolution over time. Firstly, we develop a method to automatically monitor the quality of User Stories. Secondly, we investigate the relationship between User Story quality and project performance measures such as the number of reported bugs and the occurrence of rework and delays. We measure User Story quality with the help of a recently published quality framework and tool, Automatic Quality User Story Artisan (AQUSA). For our empirical work, we use six agile open source software projects. We apply time series analysis and use the Windowed Time Lagged Cross Correlation (WTLCC) method. Our results indicate that automatic User Story quality monitoring is feasible and may result in various distinct dynamic evolution patterns. In addition, we found the following relationship patterns between User Story quality and the software development aspects. A decrease/increase in User Story quality scores is associated with (i) a decrease/increase of the number of bugs after 1–13 weeks in short-medium projects, and 12 weeks in longer ones, (ii) an increase in rework frequency after 18–28, 8–15, and 1–3 weeks for long, medium, and short projects, respectively, and (iii) an increase in delayed issues after 7–20, 8–11, and 1–3 weeks for long, medium, and short duration projects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    AQUSA Tool repository – https://github.com/gglucass/AQUSA.

References

  1. Firesmith, D.: Common requirements problems, their negative consequences, and the industry best practices to help solve them. J. Object Technol. 6(1), 17–33 (2007)

    Article  Google Scholar 

  2. Wohlin, C., et al.: Engineering and Managing Software Requirements. Springer, Heidelberg (2005). https://doi.org/10.1007/3-540-28244-0

    Book  MATH  Google Scholar 

  3. Cohn, M.: User Stories Applied: For Agile Software Development. Addison-Wesley Professional, Boston (2004)

    Google Scholar 

  4. Dingsøyr, T., Nerur, S., Balijepally, V., Moe, N.B.: A decade of agile methodologies. J. Syst. Softw. 85(6), 1213–1221 (2012)

    Article  Google Scholar 

  5. Kassab, M.: The changing landscape of requirements engineering practices over the past decade. In: 2015 IEEE 5th International Workshop on Empirical Requirements Engineering (EmpiRE), pp. 1–8. IEEE (2015)

    Google Scholar 

  6. CollabNet VersionOne: 13th Annual State of Agile Report (2018)

    Google Scholar 

  7. Wang, X., Zhao, L., Wang, Y., Sun, J.: The role of requirements engineering practices in agile development: an empirical study. In: Zowghi, D., Jin, Z. (eds.) Requirements Engineering. CCIS, vol. 432, pp. 195–209. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-43610-3_15

    Chapter  Google Scholar 

  8. Kassab, M.: An empirical study on the requirements engineering practices for agile software development. In: 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications, pp. 254–261. IEEE (2014)

    Google Scholar 

  9. Lucassen, G., Dalpiaz, F., Werf, J.M.E.M., Brinkkemper, S.: The use and effectiveness of user stories in practice. In: Daneva, M., Pastor, O. (eds.) REFSQ 2016. LNCS, vol. 9619, pp. 205–222. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30282-9_14

    Chapter  Google Scholar 

  10. Wake, B.: Invest in good stories, and smart tasks (2003)

    Google Scholar 

  11. Lucassen, G., Dalpiaz, F., van der Werf, J.M.E., Brinkkemper, S.: Improving agile requirements: the quality user story framework and tool. Requirements Eng. 21(3), 383–403 (2016)

    Article  Google Scholar 

  12. Buglione, L., Abran, A.: Improving the user story agile technique using the invest criteria. In: 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement, pp. 49–53 (2013)

    Google Scholar 

  13. Lai, S.T.: A user story quality measurement model for reducing agile software development risk. Int. J. Softw. Eng. Appl 8, 75–86 (2017)

    Google Scholar 

  14. de Souza, P.L., do Prado, A.F., de Souza, W.L., dos Santos Forghieri Pereira, S.M., Pires, L.F.: Improving agile software development with domain ontologies. In: Latifi, S. (ed.) Information Technology - New Generations. AISC, vol. 738, pp. 267–274. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77028-4_37

    Chapter  Google Scholar 

  15. Rodríguez-Pérez, G., Robles, G., Serebrenik, A., Zaidman, A., Germán, D.M., Gonzalez-Barahona, J.M.: How bugs are born: a model to identify how bugs are introduced in software components. Empir Software Eng 25, 1294–1340 (2020)

    Article  Google Scholar 

  16. Sedano, T., Ralph, P., Péraire, C.: Software development waste. In: Proceedings of the 39th International Conference on Software Engineering, ICSE 2017, pp. 130–140. IEEE Press (2017)

    Google Scholar 

  17. Tamai, T., Kamata, M.I.: Impact of requirements quality on project success or failure. In: Lyytinen, K., Loucopoulos, P., Mylopoulos, J., Robinson, B. (eds.) Design Requirements Engineering: A Ten-Year Perspective. LNBIP, vol. 14, pp. 258–275. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-540-92966-6_15

    Chapter  Google Scholar 

  18. Jahanshahi, H., Cevik, M., Başar, A.: Predicting the number of reported bugs in a software repository. In: Goutte, C., Zhu, X. (eds.) Canadian AI 2020. LNCS (LNAI), vol. 12109, pp. 309–320. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-47358-7_31

    Chapter  Google Scholar 

  19. Kai, H., Zhengwei, Q., Bo, L.: Network anomaly detection based on statistical approach and time series analysis. In: 2009 International Conference on Advanced Information Networking and Applications Workshops, pp. 205–211 (2009)

    Google Scholar 

  20. Herraiz, I., Gonzalez-Barahona, J.M., Robles, G.: Forecasting the number of changes in eclipse using time series analysis. In: 4th International Workshop on Mining Software Repositories, MSR 2007:ICSE Workshops 2007, p. 32 (2007)

    Google Scholar 

  21. Choraś, M., Kozik, R., Pawlicki, M., Hołubowicz, W., Franch, X.: Software development metrics prediction using time series methods. In: Saeed, K., Chaki, R., Janev, V. (eds.) CISIM 2019. LNCS, vol. 11703, pp. 311–323. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-28957-7_26

    Chapter  Google Scholar 

  22. Roume, C., Almurad, Z., Scotti, M., Ezzina, S., Blain, H., Delignières, D.: Windowed detrended cross-correlation analysis of synchronization processes. Phys. A 503, 1131–1150 (2018)

    Article  Google Scholar 

  23. Scott, E., Pfahl, D.: Using developers’ features to estimate story points. In: Proceedings of the 2018 International Conference on Software and System Process, pp. 106–110 (2018)

    Google Scholar 

  24. Scott, E., Charkie, K.N., Pfahl, D.: Productivity, turnover, and team stability of agile software development teams in open-source projects. In: 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE (2020)

    Google Scholar 

  25. Porru, S., Murgia, A., Demeyer, S., Marchesi, M., Tonelli, R.: Estimating story points from issue reports. In: Proceedings of the 12th International Conference on Predictive Models and Data Analytics in Software Engineering, pp. 1–10 (2016)

    Google Scholar 

  26. Boker, S.M., Rotondo, J.L., Xu, M., King, K.: Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series. Psychol. Meth. 7(3), 338 (2002)

    Article  Google Scholar 

  27. Jammazi, R., Aloui, C.: Environment degradation, economic growth and energy consumption nexus: a wavelet-windowed cross correlation approach. Phys. A 436, 110–125 (2015)

    Article  Google Scholar 

  28. Cohen, J.: A power primer. Psychol. Bull. 112(1), 155 (1992)

    Article  Google Scholar 

  29. Sigurd, B., Eeg-Olofsson, M., Van Weijer, J.: Word length, sentence length and frequency - Zipf revisited. Studia Linguistica 58(1), 37–52 (2004)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Estonian Center of Excellence in ICT research (EXCITE), ERF project TK148 IT, and by the team grant PRG 887 of the Estonian Research Council.

Author information

Authors and Affiliations

  1. Institute of Computer Science, University of Tartu, Narva mnt 18, 51009, Tartu, Estonia

    Ezequiel Scott, Tanel Tõemets & Dietmar Pfahl

Authors
  1. Ezequiel Scott
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tanel Tõemets
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dietmar Pfahl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ezequiel Scott .

Editor information

Editors and Affiliations

  1. TU Wien, Vienna, Austria

    Dietmar Winkler

  2. TU Wien, Vienna, Austria

    Stefan Biffl

  3. Blekinge Institute of Technology, Karlskrona, Sweden

    Daniel Mendez

  4. Johannes Kepler University Linz, Linz, Austria

    Manuel Wimmer

  5. Software Quality Lab GmbH, Linz, Austria

    Johannes Bergsmann

Appendix

Appendix

During the data cleaning phase, we applied the following steps:

  1. 1.

    Removal of empty rows.

  2. 2.

    Removal of special headings in the description of the user story (e.g., “h2. Back story”)

  3. 3.

    Removal of hyperlinks to web sites.

  4. 4.

    Removal of mentions to files with extensions such as “.jar”.

  5. 5.

    Removal of code examples.

  6. 6.

    Removal of different types of curly brackets combinations.

  7. 7.

    Removal of paths to files.

  8. 8.

    Removal of word whose length is longer than 19 characters. According to [29], words with more than 19 characters are very rare in English (less than 0.1%). In our case, this usually happens when the bod y of a user story describe part of the program code. For example, the string “TriggerSourceOptionsMetadata”

  9. 9.

    Removal of consecutive exclamation marks and the text between them. This notation is commonly used for adding images (e.g., “!GettingStarted.png!”)

  10. 10.

    Removal of square brackets and everything between them.

  11. 11.

    Removal of non-ASCII characters.

  12. 12.

    Removal of special characters such as “ ”, “ ”, “\({\_}\)”, and “$”.

  13. 13.

    Removal of different kinds of whitespaces (e.g., tabs, “&nbsp” etc.) and replacing them with a single whitespace.

  14. 14.

    Removal of duplicated User Stories.

  15. 15.

    Removal of upper outliers (abnormally long User Stories). Upper outliers are removed based on the description length using Turkey’s fences.

  16. 16.

    Removal of lower outliers (User Stories with less than 3 words). For example, some User Stories consisted of only the description “See: http://\(\ldots \)”.

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Scott, E., Tõemets, T., Pfahl, D. (2021). An Empirical Study of User Story Quality and Its Impact on Open Source Project Performance. In: Winkler, D., Biffl, S., Mendez, D., Wimmer, M., Bergsmann, J. (eds) Software Quality: Future Perspectives on Software Engineering Quality. SWQD 2021. Lecture Notes in Business Information Processing, vol 404. Springer, Cham. https://doi.org/10.1007/978-3-030-65854-0_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-65854-0_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-65853-3

  • Online ISBN: 978-3-030-65854-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation