Skip to main content
SpringerLink
Log in

Forecasting technical debt evolution in software systems: an empirical study

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Technical debt is considered detrimental to the long-term success of software development, but despite the numerous studies in the literature, there are still many aspects that need to be investigated for a better understanding of it. In particular, the main problems that hinder its complete understanding are the absence of a clear definition and a model for its identification, management, and forecasting. Focusing on forecasting technical debt, there is a growing notion that preventing technical debt build-up allows you to identify and address the riskiest debt items for the project before they can permanently compromise it. However, despite this high relevance, the forecast of technical debt is still little explored. To this end, this study aims to evaluate whether the quality metrics of a software system can be useful for the correct prediction of the technical debt. Therefore, the data related to the quality metrics of 8 different open-source software systems were analyzed and supplied as input to multiple machine learning algorithms to perform the prediction of the technical debt. In addition, several partitions of the initial dataset were evaluated to assess whether prediction performance could be improved by performing a data selection. The results obtained show good forecasting performance and the proposed document provides a useful approach to understanding the overall phenomenon of technical debt for practical purposes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Cunningham W. The WyCash portfolio management system. In: Proceeding of Addendum to the Proceedings on Object-Oriented Programming Systems, Languages, and Applications. 1992

  2. de Jesus J S, de Melo A C V. Technical debt and the software project characteristics. A repository-based exploratory analysis. In: Proceeding of the 19th Conference on Business Informatics (CBI). 2017, 444–453

  3. Aversano L, Bernardi M L, Cimitile M, Iammarino M, Romanyuk K. Investigating on the relationships between design smells removals and refactorings. In: Proceedings of the 15th International Conference on Software Technologies. 2020, 212–219

  4. Ardimento P, Aversano L, Bernardi M L, Cimitile M, Iammarino M. Transfer learning for just-in-time design smells prediction using temporal convolutional networks. In: Proceedings of the 16th International Conference on Software Technologies. 2021, 310–317

  5. Ardimento P, Dinapoli A. Knowledge extraction from on-line open source bug tracking systems to predict bug-fixing time. In: Proceedings of the 7th International Conference on Web Intelligence, Mining and Semantics. 2017

  6. Aversano L, Cerulo L, Palumbo C. Mining candidate web services from legacy code. In: Proceeding of the 10th International Symposium on Web Site Evolution. 2008, 37–40

  7. Aversano L, Bruno M, Di Penta M, Falanga A, Scognamiglio R. Visualizing the evolution of web services using formal concept analysis. In: Proceeding of the 8th International Workshop on Principles of Software Evolution (IWPSE’05). 2005, 57–60

  8. Rantala L, Mäntylä M. Predicting technical debt from commit contents: reproduction and extension with automated feature selection. Software Quality Journal, 2020, 28(4): 1551–1579

    Article  Google Scholar 

  9. Potdar A, Shihab E. An exploratory study on self-admitted technical debt. In: Proceeding of 2014 IEEE International Conference on Software Maintenance and Evolution. 2014, 91–100

  10. Bavota G, Russo B. A large-scale empirical study on self-admitted technical debt. In: Proceedings of the13th Working Conference on Mining Software Repositories (MSR). 2016, 315–326

  11. Maldonado E D S, Abdalkareem R, Shihab E, Serebrenik A. An empirical study on the removal of self-admitted technical debt. In: Proceeding of 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). 2017, 238–248

  12. Wehaibi S, Shihab E, Guerrouj L. Examining the impact of self-admitted technical debt on software quality. In: Proceeding of the 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER). 2016, 179–188

  13. Seaman C, Guo Y, Zazworka N, Shull F, Izurieta C, Cai Y, Vetrò A. Using technical debt data in decision making: Potential decision approaches. In: Proceedings of the 3rd International Workshop on Managing Technical Debt. 2012, 45–48

  14. Sas D, Avgeriou P. Quality attribute trade-offs in the embedded systems industry: an exploratory case study. Software Quality Journal, 2020, 28(2): 505–534

    Article  Google Scholar 

  15. Li Z, Avgeriou P, Liang P. A systematic mapping study on technical debt and its management. Journal of Systems and Software, 2015, 101: 193–220

    Article  Google Scholar 

  16. Aversano L, Bernardi M L, Cimitile M, Iammarino M, Romanyuk K. Investigating on the relationships between design smells removals and refactorings. In: Proceedings of the 15th International Conference on Software Technologies. 2020, 212–219

  17. Amanatidis T, Mittas N, Moschou A, Chatzigeorgiou A, Ampatzoglou A, Angelis L. Evaluating the agreement among technical debt measurement tools: building an empirical benchmark of technical debt liabilities. Empirical Software Engineering, 2020, 25(5): 4161–4204

    Article  Google Scholar 

  18. Halepmollasi R. A composed technical debt identification methodology to predict software vulnerabilities. In: Proceeding of the 42nd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). 2020, 186–189

  19. Spinellis D. Tool writing: A forgotten art? (software tools) IEEE Software, 2005, 22(4): 9–11

    Article  Google Scholar 

  20. Avgeriou P C, Taibi D, Ampatzoglou A, Arcelli Fontana F, Besker T, Chatzigeorgiou A, Lenarduzzi V, Martini A, Moschou A, Pigazzini I, Saarimaki N, Sas D D, de Toledo S S, Tsintzira A A. An overview and comparison of technical debt measurement tools. IEEE Software, 2021, 38(3): 61–71

    Article  Google Scholar 

  21. Knofczynski G T, Mundfrom D. Sample sizes when using multiple linear regression for prediction. Educational and Psychological Measurement, 2008, 68(3): 431–442

    Article  MathSciNet  Google Scholar 

  22. Safavian S R, Landgrebe D. A survey of decision tree classifier methodology. IEEE Transactions on Systems, Man, and Cybernetics, 1991, 21(3): 660–674

    Article  MathSciNet  Google Scholar 

  23. Dietterich T G. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 2000, 40(2): 139–157

    Article  Google Scholar 

  24. Breiman L. Random forests. Machine Learning, 2001, 45(1): 5–32

    Article  MATH  Google Scholar 

  25. Iammarino M, Zampetti F, Aversano L, Di Penta M. An empirical study on the co-occurrence between refactoring actions and self-admitted technical debt removal. Journal of Systems and Software, 2021, 178: 110976

    Article  Google Scholar 

  26. Iammarino M, Zampetti F, Aversano L, Di Penta M. Self-admitted technical debt removal and refactoring actions: Co-occurrence or more? In: Proceeding of 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME). 2019, 186–190

  27. Aversano L, Iammarino M, Carapella M, Vecchio A D, Nardi L. On the relationship between self-admitted technical debt removals and technical debt measures. Algorithms, 2020, 13(7): 168

    Article  Google Scholar 

  28. Li Z, Liang P, Avgeriou P, Guelf N, Ampatzoglou A. An empirical investigation of modularity metrics for indicating architectural technical debt. In: Proceedings of the 10th International ACM Sigsoft Conference on Quality of Software Architectures. 2014, 119–128

  29. Ampatzoglou A, Michailidis A, Sarikyriakidis C, Ampatzoglou A, Chatzigeorgiou A, Avgeriou P. A framework for managing interest in technical debt: An industrial validation. In: Proceedings of 2018 International Conference on Technical Debt. 2018, 115–124

  30. Digkas G, Lungu M, Chatzigeorgiou A, Avgeriou P. The evolution of technical debt in the apache ecosystem. In: Lopes A, de Lemos R, eds. Software Architecture. Cham: Springer, 2017, 51–66

    Chapter  Google Scholar 

  31. Chatzigeorgiou A, Ampatzoglou A, Ampatzoglou A, Amanatidis T. Estimating the breaking point for technical debt. In: Proceeding of the 7th International Workshop on Managing Technical Debt (MTD). 2015, 53–56

  32. Skourletopoulos G, Mavromoustakis C X, Bahsoon R, Mastorakis G, Pallis E. Predicting and quantifying the technical debt in cloud software engineering. In: Proceeding of the 19th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD). 2014, 36–40

  33. Arvanitou E M, Ampatzoglou A, Bibi S, Chatzigeorgiou A, Stamelos I. Monitoring technical debt in an industrial setting. In: Proceedings of Evaluation and Assessment on Software Engineering. 2019, 123–132

  34. Zazworka N, Shaw M A, Shull F, Seaman C. Investigating the impact of design debt on software quality. In: Proceedings of the 2nd Workshop on Managing Technical Debt. 2011, 17–23

  35. Zazworka N, Spínola R O, Vetro’ A, Shull F, Seaman C. A case study on effectively identifying technical debt. In: Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering. 2013, 42–47

  36. Bellomo S, Nord R L, Ozkaya I, Popeck M. Got technical debt? Surfacing elusive technical debt in issue trackers. In: Proceedings of the 13th Working Conference on Mining Software Repositories (MSR). 2016, 327–338

  37. Tsoukalas D, Kehagias D, Siavvas M, Chatzigeorgiou A. Technical debt forecasting: An empirical study on open-source repositories. Journal of Systems and Software, 2020, 170: 110777

    Article  Google Scholar 

  38. Aversano L, Bernardi M L, Cimitile M, Iammarino M. Technical debt predictive model through temporal convolutional network. In: Proceeding of 2021 International Joint Conference on Neural Networks (IJCNN). 2021, 1–8

  39. Saarimaki N, Baldassarre M T, Lenarduzzi V, Romano S. On the accuracy of SonarQube technical debt remediation time. In: Proceeding of the 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). 2019, 317–324

  40. Letouzey J L. The SQALE method for evaluating technical debt. In: Proceedings of the 3rd International Workshop on Managing Technical Debt. 2012, 31–36

  41. Hitz M, Montazeri B. Chidamber and Kemerer’s metrics suite: a measurement theory perspective. IEEE Transactions on Software Engineering, 1996, 22(4): 267–271

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Engineering, University of Sannio, Benevento, 82100, Italy

    Lerina Aversano, Mario Luca Bernardi, Martina Iammarino & Debora Montano

  2. Department of Law and Economics, Unitelma Sapienza University of Rome, Rome, 00161, Italy

    Marta Cimitile

Authors
  1. Lerina Aversano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mario Luca Bernardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marta Cimitile
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Martina Iammarino
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Debora Montano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lerina Aversano.

Additional information

Lerina Aversano is an associate professor at the Department of Engineering of the University of Sannio, Italy. She received the PhD in Computer Engineering in July 2003 at the same University and was assistant professor from 2005. She also was a research leader at RCOST — Research Centre On Software Technology — of the University of Sannio, Italy from 2005. Her research interests include software maintenance, program comprehension, reverse engineering, reengineering, migration, business process modeling, business process evolution, software system evolution, software quality.

Mario Luca Bernardi received the PhD degree in Information Engineering from the University of Sannio, Italy in 2007. He is currently an assistant professor of Computer Science at the University of Sannio, Italy. Since 2003 he has been a researcher in the field of software engineering and his list of publications contains more than 100 papers published in journals and conference proceedings. His main research interests include artificial intelligence applied to several domains, pervasive computing and software engineering. He serves both as a member of the program and organizing committees of conferences, and as associate editor and reviewer of papers submitted to some of the main journals and magazines in the field of software engineering, software maintenance and program comprehension.

Marta Cimitile is an associate professor at Unitelma Sapienza University of Rome, Italy. She received a PhD in Computer Science in 2008 at the Department of Computer Science at the University of Bari, Italy. She published more than 100 papers at international conferences and journals. Her main research topics are: Process and Data Mining, Artificial Intelligence. She was involved in several industrial and research projects and she is a founding member of the SpinOff of University of Bari named Software Engineering Research and Practices s.r.l. She was in the Editorial Board of several international journals like Computational Intelligence and Neuroscience.

Martina Iammarino is a PhD student in Computer Engineering. She received master’s degree in Computer Engineering at the University of Sannio, Italy. Her research interests include software quality, software metrics, refactoring, modeling, evaluation and evolution, technical debt, and software measurement.

Debora Montano has a master’s degree in Statistics and Actuarial Sciences at the University of Sannio, Italy. She is currently researching at the Computer Science Department of the University of Sannio, Italy where she carries out research as a Data Analyst for Machine Learning and Big Data processing activities. Debora has already worked as a researcher at the same University, where she worked on statistical learning methods for predictive maintenance. Currently, Debora’s research fields are technical debt, software quality, refactoring, machine learning, deep learning, and empirical studies in the medical field.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aversano, L., Bernardi, M.L., Cimitile, M. et al. Forecasting technical debt evolution in software systems: an empirical study. Front. Comput. Sci. 17, 173210 (2023). https://doi.org/10.1007/s11704-022-1541-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-022-1541-7

Keywords

Search

Navigation