skip to main content
10.1145/3644384.3644471acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Increasing, not Diminishing: Investigating the Returns of Highly Maintainable Code

Published: 07 June 2024 Publication History

Abstract

Understanding and effectively managing Technical Debt (TD) remains a vital challenge in software engineering. While many studies on code-level TD have been published, few illustrate the business impact of low-quality source code. In this study, we combine two publicly available datasets to study the association between code quality on the one hand, and defect count and implementation time on the other hand. We introduce a value-creation model, derived from regression analyses, to explore relative changes from a baseline. Our results show that the associations vary across different intervals of code quality. Furthermore, the value model suggests strong non-linearities at the extremes of the code quality spectrum. Most importantly, the model suggests amplified returns on investment in the upper end. We discuss the findings within the context of the "broken windows" theory and recommend organizations to diligently prevent the introduction of code smells in files with high churn. Finally, we argue that the value-creation model can be used to initiate discussions regarding the return on investment in refactoring efforts.

References

[1]
Claudia Ayala, Burak Turhan, Xavier Franch, and Natalia Juristo. 2022. Use and Misuse of the Term "Experiment" in Mining Software Repositories Research. IEEE Trans. on Software Engineering 48, 11 (2022), 4229--4248.
[2]
Robert Baggen, José Pedro Correia, Katrin Schill, and Joost Visser. 2012. Standardized Code Quality Benchmarking for Improving Software Maintainability. Software Quality Journal 2 (2012), 287--307.
[3]
Markus Borg, Ilyana Pruvost, Enys Mones, and Adam Tornhill. 2024. Accompanying Replication Package. https://zenodo.org/records/10560722
[4]
Markus Borg, Adam Tornhill, and Enys Mones. 2023. U Owns the Code That Changes and How Marginal Owners Resolve Issues Slower in Low-Quality Source Code. In Proc. of the 27th Int'l. Conf. on Evaluation and Assessment in Software Engineering. 368--377.
[5]
Preetha Chatterjee, Tushar Sharma, and Paul Ralph. 2022. Empirical Standards for Repository Mining. In Proc. of the 19th Int'l. Conf. on Mining Software Repositories. 142--143.
[6]
B. Efron. 1979. Bootstrap Methods: Another Look at the Jackknife. The Annals of Statistics 7, 1 (1979), 1--26.
[7]
David Farley. 2021. Modern Software Engineering: Doing What Works to Build Better Software Faster. Addison-Wesley Professional, Boston, MA, USA.
[8]
Norman Fenton. 1994. Software Measurement: A Necessary Scientific Basis. IEEE Trans. on Software Engineering 20, 3 (1994), 199--206.
[9]
Nicole Forsgren, Jez Humble, and Gene Kim. 2018. Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations (1 ed.). IT Revolution Press, Portland, Oregon.
[10]
Nicole Forsgren, Margaret-Anne Storey, Chandra Maddila, Thomas Zimmermann, Brian Houck, and Jenna Butler. 2021. The SPACE of Developer Productivity: There's more to it than you think. Queue 19, 1 (2021), 10:20--10:48.
[11]
Matthieu Foucault, Jean-Rémy Falleri, and Xavier Blanc. 2014. Code Ownership in Open-Source Software. In Proc. of the 18th Int'l. Conf. on Evaluation and Assessment in Software Engineering. 1--9.
[12]
Emilio Gianicolo, Martin Eichler, Oliver Muensterer, Konstantin Strauch, and Maria Blettner. 2020. Methods for Evaluating Causality in Observational Studies. Deutsches Arzteblatt Int'l. 117, 7 (2020), 101--107.
[13]
Emanuel Giger, Martin Pinzger, and Harald Gall. 2011. Comparing Fine-grained Source Code Changes and Code Churn for Bug Prediction. In Proc. of the 8th Working Conf. on Mining Software Repositories. 83--92.
[14]
Tracy Hall, Sarah Beecham, David Bowes, David Gray, and Steve Counsell. 2012. A Systematic Literature Review on Fault Prediction Performance in Software Engineering. IEEE Trans. on Software Engineering 38, 6 (2012), 1276--1304.
[15]
Kim Herzig, Sascha Just, and Andreas Zeller. 2013. It's Not a Bug, It's a Feature: How Misclassification Impacts Bug Prediction. In Proc. of the 35th Int'l. Conf. on Software Engineering. 392--401.
[16]
Seyedrebvar Hosseini, Burak Turhan, and Dimuthu Gunarathna. 2019. A Systematic Literature Review and Meta-Analysis on Cross Project Defect Prediction. IEEE Trans. on Software Engineering 45, 2 (2019), 111--147.
[17]
Andrew Hunt and David Thomas. 1999. The Pragmatic Programmer: From Journeyman to Master (1 ed.). Addison-Wesley Professional, Boston, MA, USA.
[18]
Helvio Jeronimo Junior and Guilherme Horta Travassos. 2022. Consolidating a Common Perspective on Technical Debt and its Management Through a Tertiary Study. Information and Software Technology 149 (2022), 106964.
[19]
Barbara Kitchenham, Lech Madeyski, David Budgen, Jacky Keung, Pearl Brereton, Stuart Charters, Shirley Gibbs, and Amnart Pohthong. 2017. Robust Statistical Methods for Empirical Software Engineering. Empirical Software Engineering 22, 2 (2017), 579--630.
[20]
Michael Klas, Adam Trendowicz, Yasushi Ishigai, and Haruka Nakao. 2011. Handling Estimation Uncertainty with Bootstrapping: Empirical Evaluation in the Context of Hybrid Prediction Methods. In Proc. of the Int'l. Symposium on Empirical Software Engineering and Measurement. 245--254.
[21]
Gunes Koru, Dongsong Zhang, Khaled El Emam, and Hongfang Liu. 2009. An Investigation into the Functional Form of the Size-Defect Relationship for Software Modules. IEEE Trans. on Software Engineering 35, 2 (2009), 293--304.
[22]
Guilherme Lacerda, Fabio Petrillo, Marcelo Pimenta, and Yann Gael Gueheneuc. 2020. Code Smells and Refactoring: A Tertiary Systematic Review of Challenges and Observations. Journal of Systems and Software 167 (2020), 110610.
[23]
Valentina Lenarduzzi, Terese Besker, Davide Taibi, Antonio Martini, and Francesca Arcelli Fontana. 2021. A Systematic Literature Review on Technical Debt Prioritization: Strategies, Processes, Factors, and Tools. Journal of Systems and Software 171 (2021), 110827.
[24]
William Leven, Hampus Broman, Terese Besker, and Richard Torkar. 2023. Testing the Broken Windows Theory in the context of Technical Debt.
[25]
Wei Li and Raed Shatnawi. 2007. An Empirical Study of the Bad Smells and Class Error Probability in the Post-Release Object-Oriented System Evolution. Journal of Systems and Software 80, 7 (2007), 1120--1128.
[26]
Alberto S. Nunez-Varela, Hector G. Perez-Gonzalez, Francisco E. Martinez-Perez, and Carlos Soubervielle-Montalvo. 2017. Source Code Metrics: A Systematic Mapping Study. Journal of Systems and Software 128 (2017), 164--197.
[27]
Steffen Olbrich, Daniela S. Cruzes, Victor Basili, and Nico Zazworka. 2009. The Evolution and Impact of Code Smells: A Case Study of Two Open Source Systems. In Proc. of the 3rd Int'l. Symposium on Empirical Software Engineering and Measurement. 390--400.
[28]
Judea Pearl. 2018. The Book of Why: The New Science of Cause and Effect (1 ed.). Ingram Publisher Services, La Vergne, TN, USA.
[29]
Danijel Radjenovic, Marjan Hericko, Richard Torkar, and Ales Zivkovic. 2013. Software Fault Prediction Metrics: A Systematic Literature Review. Information and Software Technology 55, 8 (2013), 1397--1418.
[30]
Mehwish Riaz, Emilia Mendes, and Ewan Tempero. 2009. A Systematic Review of Software Maintainability Prediction and Metrics. In Proc. of the 3rd Int'l. Symposium on Empirical Software Engineering and Measurement. 367--377.
[31]
Nicolli Rios, Manoel Gomes de Mendonca Neto, and Rodrigo Oliveira Spinola. 2018. A Tertiary Study on Technical Debt: Types, Management Strategies, Research Trends, and Base Information for Practitioners. Information and Software Technology 102 (2018), 117--145.
[32]
Jose Amancio M. Santos, Joao B. Rocha-Junior, Luciana Carla Lins Prates, Rogeres Santos do Nascimento, Mydia Falcao Freitas, and Manoel Gomes de Mendonca. 2018. A Systematic Review on the Code Smell Effect. Journal of Systems and Software 144 (2018), 450--477.
[33]
Dag Sjoberg and Gunnar Rye Bergersen. 2023. Improving the Reporting of Threats to Construct Validity. In Proc. of the 27th Int'l. Conf. on Evaluation and Assessment in Software Engineering. 205--209.
[34]
Dag I.K. Sjoberg, Aiko Yamashita, Bente C.D. Anda, Audris Mockus, and Tore Dyba. 2013. Quantifying the Effect of Code Smells on Maintenance Effort. IEEE Trans. on Software Engineering 39, 8 (2013), 1144--1156.
[35]
Zephyrin Soh, Aiko Yamashita, Foutse Khomh, and Yann-Gael Gueheneuc. 2016. Do Code Smells Impact the Effort of Different Maintenance Programming Activities?. In Proc. of the 23rd Int'l. Conf. on Software Analysis, Evolution, and Reengineering, Vol. 1. 393--402.
[36]
Vali Tawosi, Rebecca Moussa, and Federica Sarro. 2022. On the Relationship Between Story Points and Development Effort in Agile Open-Source Software. In Proc. of the 16th Int'l. Symposium on Empirical Software Engineering and Measurement. 183--194.
[37]
Adam Tornhill. 2018. Software Design X-Rays: Fix Technical Debt with Behavioral Code Analysis. Pragmatic Bookshelf.
[38]
Adam Tornhill and Markus Borg. 2022. Code Red: The Business Impact of Code Quality - A Quantitative Study of 39 Proprietary Production Codebases. In Proc. of the 5th Int'l. Conf. on Technical Debt. 11--20.
[39]
Zeynep Tufekci. 2022. The Shameful Open Secret Behind Southwest's Failure. The New York Times (Dec. 2022). https://www.nytimes.com/2022/12/31/opinion/southwest-airlines-computers.html
[40]
James Wilson and George Kelling. 1982. Broken Windows: The Police and Neighborhood Safety. Atlantic (1982), 29--38.
[41]
Aiko Yamashita and Leon Moonen. 2012. Do Code Smells Reflect Important Maintainability Aspects?. In Proc. of the 28th IEEE Int'l. Conf. on Software Maintenance. 306--315.
[42]
Nico Zazworka, Michele A. Shaw, Forrest Shull, and Carolyn Seaman. 2011. Investigating the Impact of Design Debt on Software Quality. In Proc. of the 2nd Workshop on Managing Technical Debt. 17--23.
[43]
Xiaofang Zhang, Yida Zhou, and Can Zhu. 2017. An Empirical Study of the Impact of Bad Designs on Defect Proneness. In Proc. of the Int'l. Conf. on Software Analysis, Testing and Evolution. 1--9.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
TechDebt '24: Proceedings of the 7th ACM/IEEE International Conference on Technical Debt
April 2024
55 pages
ISBN:9798400705908
DOI:10.1145/3644384
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 June 2024

Check for updates

Author Tags

  1. mining software repositories
  2. source code quality
  3. maintainability
  4. technical debt
  5. business impact

Qualifiers

  • Research-article

Conference

TechDebt '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 14 of 31 submissions, 45%

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 76
    Total Downloads
  • Downloads (Last 12 months)76
  • Downloads (Last 6 weeks)10
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media