Skip to main content
SpringerLink
Log in

An Analysis of Automated Technical Debt Measurement

  • Conference paper
  • First Online:
Enterprise Information Systems (ICEIS 2019)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 378))

Included in the following conference series:

Abstract

Background: Measuring and understanding Technical Debt (TD) is quite complex since there are a number of different definitions and techniques that have been proposed in the last few years and it is not clear which ones should be used in which conditions. The approaches proposed are almost never based on the existing ones and their validation is often performed in a very limited number of projects. For this reasons, practitioners are confused and find difficult to apply such approaches in their projects.

Goals: This paper investigates the available techniques for evaluating TD using automated tools aiming at helping practitioners and researcher in understanding the available options and apply them correctly.

Method: The study has been performed as a Systematic Literature Review (SLR) applied to 835 studies obtained from the three largest digital libraries and databases.

Results: After applying all filtering stages, 38 papers out of 835 have been selected and analyzed in depth. Almost all of them propose novel approaches to measure TD using different criteria and they do not extend or validate existing approaches.

Conclusions: The area is not mature and it lacks independent evaluations of the models proposed. Authors focus on proposing new approaches and no consolidation can be identified. Moreover, almost all the approaches proposed are automated only partially and through prototype tools designed just to support the studies analyzed in the paper in which the approach is proposed and rarely maintained. These facts makes difficult the application of such methods by practitioners.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.sqale.org/.

References

  1. Scientific toolworks, inc. understand 2.6. http://www.scitools.com/

  2. de Almeida, R.R., Treude, C., Kulesza, U.: Tracy: a business-driven technical debt prioritization framework. In: 35th International Conference on Software Maintenance and Evolution (ICSME 2019) (2019)

    Google Scholar 

  3. Alves, N.S., Mendes, T.S., de Mendonça, M.G., Spínola, R.O., Shull, F., Seaman, C.: Identification and management of technical debt: a systematic mapping study. Inf. Softw. Technol. 70, 100–121 (2016)

    Article  Google Scholar 

  4. Ampatzoglou, A., Ampatzoglou, A., Avgeriou, P., Chatzigeorgiou, A.: Establishing a framework for managing interest in technical debt. In: 5th International Symposium on Business Modeling and Software Design, BMSD (2015)

    Google Scholar 

  5. Ampatzoglou, A., Ampatzoglou, A., Chatzigeorgiou, A., Avgeriou, P.: The financial aspect of managing technical debt: a systematic literature review. Inf. Softw. Technol. 64, 52–73 (2015)

    Article  Google Scholar 

  6. Arcelli-Fontana, F., Pigazzini, I., Roveda, R., Tamburri, D., Zanoni, M., Nitto, E.: ARCAN: a tool for architectural smells detection. In: Proceeding of the International Conference on Software Architecture, ICSA 2017, pp. 282–285. IEEE (2017)

    Google Scholar 

  7. Arvedahl, S.: Introducing debtgrep, a tool for fighting technical debt in base station software. In: Proceedings of the 2018 International Conference on Technical Debt TechDebt 2018, pp. 51–52. ACM (2018)

    Google Scholar 

  8. Ayewah, N., Hovemeyer, D., Morgenthaler, J.D., Penix, J., Pugh, W.: Using static analysis to find bugs. IEEE Softw. 25(5), 22–29 (2008)

    Article  Google Scholar 

  9. Behutiye, W.N., Rodríguez, P., Oivo, M., Tosun, A.: Analyzing the concept of technical debt in the context of agile software development: a systematic literature review. Inf. Softw. Technol. 82, 139–158 (2017)

    Article  Google Scholar 

  10. BenIdris, M., Ammar, H., Dzielski, D.: Investigate, identify and estimate the technical debt: a systematic mapping study. Int. J. Softw. Eng. Appl. 9(5), 1–14 (2018)

    Google Scholar 

  11. Besker, T., Martini, A., Bosch, J.: A systematic literature review and a unified model of ATD. In: 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 189–197. IEEE (2016)

    Google Scholar 

  12. Besker, T., Martini, A., Bosch, J.: Managing architectural technical debt: a unified model and systematic literature review. J. Syst. Softw. 135, 1–16 (2018)

    Article  Google Scholar 

  13. Biaggi, A., Arcelli Fontana, F., Roveda, R.: An architectural smells detection tool for C and C++ projects. In: 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 417–420 (2018)

    Google Scholar 

  14. Bougouffa, S., Dong, Q.H., Diehm, S., Gemein, F., Vogel-Heuser, B.: Technical debt indication in plc code for automated production systems: Introducing a domain specific static code analysis tool. IFAC-Papers OnLine 51(10), 70–75 (2018). 3rd IFAC Conference on Embedded Systems, Computational Intelligence and Telematics in Control CESCIT 2018

    Article  Google Scholar 

  15. Britsman, E., Tanriverdi, Ö.: Identifying technical debt impact on maintenance effort-an industrial case study (2015). https://gupea.ub.gu.se/handle/2077/40110

  16. Capitán, L., Vogel-Heuser, B.: Metrics for software quality in automated production systems as an indicator for technical debt. In: 2017 13th IEEE Conference on Automation Science and Engineering (CASE), pp. 709–716 (2017)

    Google Scholar 

  17. Capitán, L., Vogel-Heuser, B.: Metrics for software quality in automated production systems as an indicator for technical debt. In: 2017 13th IEEE Conference on Automation Science and Engineering (CASE), pp. 709–716, August 2017

    Google Scholar 

  18. Chatzigeorgiou, A., Ampatzoglou, A., Ampatzoglou, A., Amanatidis, T.: Estimating the breaking point for technical debt. In: 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD), pp. 53–56. IEEE (2015)

    Google Scholar 

  19. Ciolkowski, M., Guzmán, L., Trendowicz, A., Salfner, F.: Lessons learned from the ProDebt research project on planning technical debt strategically. In: Felderer, M., Méndez Fernández, D., Turhan, B., Kalinowski, M., Sarro, F., Winkler, D. (eds.) PROFES 2017. LNCS, vol. 10611, pp. 523–534. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69926-4_42

    Chapter  Google Scholar 

  20. Coman, I.D., Sillitti, A., Succi, G.: Investigating the usefulness of pair-programming in a mature agile team. In: 9th International Conference on eXtreme Programming and Agile Processes in Software Engineering (XP2008), June 2008

    Google Scholar 

  21. Corral, L., Sillitti, A., Succi, G.: Software development processes for mobile systems: is agile really taking over the business? In: 1st International Workshop on Mobile-Enabled Systems (MOBS 2013) at ICSE 2013, June 2013

    Google Scholar 

  22. Cunningham, W.: The WyCash portfolio management system. In: Addendum to the Proceedings on Object-Oriented Programming Systems, Languages, and Applications (Addendum), OOPSLA 1992, Vancouver, British Columbia, Canada, pp. 29–30. Association for Computing Machinery, New York (1992). https://doi.org/10.1145/157709.157715

  23. Curtis, B., Sappidi, J., Szynkarski, A.: Estimating the size, cost, and types of technical debt. In: Proceedings of the Third International Workshop on Managing Technical Debt, pp. 49–53. IEEE Press (2012)

    Google Scholar 

  24. Falessi, D., Reichel, A.: Towards an open-source tool for measuring and visualizing the interest of technical debt. In: 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD), pp. 1–8. IEEE (2015)

    Google Scholar 

  25. Falessi, D., Shaw, M.A., Shull, F., Mullen, K., Keymind, M.S.: Practical considerations, challenges, and requirements of tool-support for managing technical debt. In: 2013 4th International Workshop on Managing Technical Debt (MTD), pp. 16–19. IEEE (2013)

    Google Scholar 

  26. Flisar, J., Podgorelec, V.: Identification of self-admitted technical debt using enhanced feature selection based on word embedding. IEEE Access 7, 106475–106494 (2019)

    Article  Google Scholar 

  27. Gaudin, O.: Evaluate your technical debt with sonar. Sonar, June 2009

    Google Scholar 

  28. Griffith, I., Reimanis, D., Izurieta, C., Codabux, Z., Deo, A., Williams, B.: The correspondence between software quality models and technical debt estimation approaches. In: 2014 Sixth International Workshop on Managing Technical Debt (MTD), pp. 19–26. IEEE (2014)

    Google Scholar 

  29. Gruber, H., Plösch, R., Saft, M.: On the validity of benchmarking for evaluating code quality. In: IWSM/MENSURA 2010 (2010). https://www.iwsm-mensura.org/2010-conference/

  30. Heitlager, I., Kuipers, T., Visser, J.: A practical model for measuring maintainability. In: 6th International Conference on the Quality of Information and Communications Technology, QUATIC 2007, pp. 30–39. IEEE (2007)

    Google Scholar 

  31. Izurieta, C., Griffith, I., Reimanis, D., Luhr, R.: On the uncertainty of technical debt measurements. In: 2013 International Conference on Information Science and Applications (ICISA), pp. 1–4. IEEE (2013)

    Google Scholar 

  32. Kamei, Y., Maldonado, E.D.S., Shihab, E., Ubayashi, N.: Using analytics to quantify interest of self-admitted technical debt. In: QuASoQ/TDA@ APSEC, pp. 68–71 (2016)

    Google Scholar 

  33. Khomyakov, I., Makhmutov, Z., Mirgalimova, R., Sillitti, A.: Automated measurement of technical debt: a systematic literature review. In: 21st International Conference on Enterprise Information Systems (ICEIS 2019), May 2019

    Google Scholar 

  34. Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering (version 2.3). Technical report, Keele University and University of Durham (2007)

    Google Scholar 

  35. Kumar, S., Bahsoon, R., Chen, T., Buyya, R.: Identifying and estimating technical debt for service composition in SaaS cloud. In: 2019 IEEE International Conference on Web Services (ICWS), pp. 121–125, July 2019

    Google Scholar 

  36. Lavazza, L., Morasca, S., Tosi, D.: A method to optimize technical debt management in timed-boxed processes. In: The Thirteenth International Conference on Software Engineering Advances (ICSEA 2018) (2018)

    Google Scholar 

  37. Lenarduzzi, V., Sillitti, A., Taibi, D.: Analyzing forty years of software maintenance models. In: 39th International Conference on Software Engineering (ICSE 2017), May 2017

    Google Scholar 

  38. Lenarduzzi, V., Martini, A., Taibi, D., Tamburri, D.A.: Towards surgically-precise technical debt estimation: early results and research roadmap. In: Proceedings of the 3rd ACM SIGSOFT International Workshop on Machine Learning Techniques for Software Quality Evaluation, MaLTeSQuE 2019, pp. 37–42. ACM (2019)

    Google Scholar 

  39. Lenarduzzi, V., Saarimäki, N., Taibi, D.: The technical debt dataset. In: Proceedings of the Fifteenth International Conference on Predictive Models and Data Analytics in Software Engineering PROMISE 2019, pp. 2–11. ACM (2019)

    Google Scholar 

  40. Letouzey, J.L.: The SQALE method for evaluating technical debt. In: 2012 Third International Workshop on Managing Technical Debt (MTD), pp. 31–36. IEEE (2012)

    Google Scholar 

  41. Letouzey, J.L., Ilkiewicz, M.: Managing technical debt with the SQALE method. IEEE Softw. 29(6), 44–51 (2012)

    Article  Google Scholar 

  42. Li, Z., Avgeriou, P., Liang, P.: A systematic mapping study on technical debt and its management. J. Syst. Softw. 101, 193–220 (2015)

    Article  Google Scholar 

  43. Li, Z., Liang, P., Avgeriou, P., Guelfi, N., Ampatzoglou, A.: An empirical investigation of modularity metrics for indicating architectural technical debt. In: Proceedings of the 10th international ACM Sigsoft conference on Quality of Software Architectures, pp. 119–128. ACM (2014)

    Google Scholar 

  44. Luhr, R.L., et al.: The application of technical debt mitigation techniques to a multidisciplinary software project. Ph.D. thesis, Montana State University-Bozeman, College of Engineering (2015)

    Google Scholar 

  45. Maldonado, E.D.S., Shihab, E.: Detecting and quantifying different types of self-admitted technical debt. In: 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD), pp. 9–15. IEEE (2015)

    Google Scholar 

  46. Marinescu, R.: Assessing technical debt by identifying design flaws in software systems. IBM J. Res. Dev. 56(5), 9–1 (2012)

    Article  Google Scholar 

  47. Martini, A., Bosch, J., Chaudron, M.: Architecture technical debt: understanding causes and a qualitative model. In: Proceedings of the 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications, SEAA 2014, pp. 85–92. IEEE (2014)

    Google Scholar 

  48. Martini, A., Sikander, E., Madlani, N.: A semi-automated framework for the identification and estimation of architectural technical debt: a comparative case-study on the modularization of a software component. Inf. Softw. Technol. 93, 264–279 (2018)

    Article  Google Scholar 

  49. Mayr, A., Plösch, R., Körner, C.: A benchmarking-based model for technical debt calculation. In: 2014 14th International Conference on Quality Software (QSIC), pp. 305–314. IEEE (2014)

    Google Scholar 

  50. Mendes, T.S., Gomes, F.G.S., Gonçalves, D.P., Mendonça, M.G., Novais, R.L., Spínola, R.O.: VisminerTD: a tool for automatic identification and interactive monitoring of the evolution of technical debt items. J. Brazil. Comput. Soc. 25(1), 2 (2019)

    Article  Google Scholar 

  51. Monteith, J.Y., McGregor, J.D.: Exploring software supply chains from a technical debt perspective. In: Proceedings of the 4th International Workshop on Managing Technical Debt, pp. 32–38. IEEE Press (2013)

    Google Scholar 

  52. Nugroho, A., Visser, J., Kuipers, T.: An empirical model of technical debt and interest. In: Proceedings of the 2nd Workshop on Managing Technical Debt, pp. 1–8. ACM (2011)

    Google Scholar 

  53. Pacheco, A., Marín-Raventós, G., López, G.: Designing a technical debt visualization tool to improve stakeholder communication in the decision-making process: a case study. In: Tjoa, A.M., Raffai, M., Doucek, P., Novak, N.M. (eds.) CONFENIS 2018. LNBIP, vol. 327, pp. 15–26. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99040-8_2

    Chapter  Google Scholar 

  54. Parodi, E., Matalonga, S., Macchi, D., Solari, M.: Comparing technical debt in student exercises using test driven development, test last and ad hoc programming. In: 2016 XLII Latin American Computing Conference (CLEI), pp. 1–10. IEEE (2016)

    Google Scholar 

  55. Parthiban, D.G.: Examination of tools for managing different dimensions of technical debt. CoRR abs/1904.11062 (2019). http://arxiv.org/abs/1904.11062

  56. Pecorelli, F., Di Nucci, D., De Roover, C., De Lucia, A.: On the role of data balancing for machine learning-based code smell detection. In: Proceedings of the 3rd ACM SIGSOFT International Workshop on Machine Learning Techniques for Software Quality Evaluation, MaLTeSQuE 2019, pp. 19–24. ACM (2019)

    Google Scholar 

  57. Ploesch, R., Gruber, H., Pomberger, G., Saft, M., Schiffer, S.: Tool support for expert-centred code assessments. In: 2008 1st International Conference on Software Testing, Verification, and Validation, pp. 258–267. IEEE (2008)

    Google Scholar 

  58. Poliakov, D., et al.: A systematic mapping study on technical debt definition (2015). https://pdfs.semanticscholar.org/a425/2d6a5d9522a85984379f8ee3bf118df782c1.pdf

  59. Potdar, A., Shihab, E.: An exploratory study on self-admitted technical debt. In: 2014 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 91–100. IEEE (2014)

    Google Scholar 

  60. Ramasubbu, N., Kemerer, C.F.: Integrating technical debt management and software quality management processes: a normative framework and field tests. IEEE Trans. Software Eng. 45(3), 285–300 (2019)

    Article  Google Scholar 

  61. Ren, X., Xing, Z., Xia, X., Lo, D., Wang, X., Grundy, J.: Neural network-based detection of self-admitted technical debt: from performance to explainability. ACM Trans. Softw. Eng. Methodol. 28(3), 15:1–15:45 (2019)

    Article  Google Scholar 

  62. Ribeiro, L.F., de Freitas Farias, M.A., Mendonça, M.G., Spínola, R.O.: Decision criteria for the payment of technical debt in software projects: a systematic mapping study. In: ICEIS, no. 1, pp. 572–579 (2016)

    Google Scholar 

  63. Rios, N., de Mendonça Neto, M.G., Spínola, R.O.: A tertiary study on technical debt: types, management strategies, research trends, and base information for practitioners. Inf. Softw. Technol. 102, 117–145 (2018)

    Article  Google Scholar 

  64. Sierra, G., Shihab, E., Kamei, Y.: A survey of self-admitted technical debt. J. Syst. Softw. 152, 70–82 (2019)

    Article  Google Scholar 

  65. Singh, V., Pollock, L.L., Snipes, W., Kraft, N.A.: A case study of program comprehension effort and technical debt estimations. In: 2016 IEEE 24th International Conference on Program Comprehension (ICPC), pp. 1–9. IEEE (2016)

    Google Scholar 

  66. Singh, V., Snipes, W., Kraft, N.A.: A framework for estimating interest on technical debt by monitoring developer activity related to code comprehension. In: 2014 Sixth International Workshop on Managing Technical Debt (MTD), pp. 27–30. IEEE (2014)

    Google Scholar 

  67. Skiada, P., Ampatzoglou, A., Arvanitou, E., Chatzigeorgiou, A., Stamelos, I.: Exploring the relationship between software modularity and technical debt. In: 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 404–407 (2018)

    Google Scholar 

  68. Skourletopoulos, G., Mavromoustakis, C.X., Mastorakis, G., Rodrigues, J.J., Chatzimisios, P., Batalla, J.M.: A fluctuation-based modelling approach to quantification of the technical debt on mobile cloud-based service level. In: 2015 IEEE Globecom Workshops (GC Wkshps), pp. 1–6. IEEE (2015)

    Google Scholar 

  69. Snipes, W., Nair, A.R., Murphy-Hill, E.: Experiences gamifying developer adoption of practices and tools. In: Companion Proceedings of the 36th International Conference on Software Engineering, pp. 105–114. ACM (2014)

    Google Scholar 

  70. Soudris, D., et al.: Exa2pro programming environment: architecture and applications. In: Proceedings of the 18th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation, SAMOS 2018, pp. 202–209. ACM (2018)

    Google Scholar 

  71. Spínola, R.O., Zazworka, N., Vetro, A., Shull, F., Seaman, C.: Understanding automated and human-based technical debt identification approaches-a two-phase study. J. Brazil. Comput. Soc. 25(1), 5 (2019)

    Article  Google Scholar 

  72. Tom, E., Aurum, A., Vidgen, R.: An exploration of technical debt. J. Syst. Softw. 86(6), 1498–1516 (2013)

    Article  Google Scholar 

  73. Tsantalis, N., Chaikalis, T., Chatzigeorgiou, A.: Jdeodorant: identification and removal of type-checking bad smells. In: 12th European Conference on Software Maintenance and Reengineering, CSMR 2008, pp. 329–331. IEEE (2008)

    Google Scholar 

  74. Tsintzira, A.A., Ampatzoglou, A., Matei, O., Ampatzoglou, A., Chatzigeorgiou, A., Heb, R.: Technical debt quantification through metrics: an industrial validation. In: 15th China-Europe International Symposium on Software Engineering Education (CEISEE 2019). IEEE (2019)

    Google Scholar 

  75. Tsoukalas, D., Siavvas, M., Jankovic, M., Kehagias, D., Chatzigeorgiou, A., Tzovaras, D.: Methods and tools for td estimation and forecasting: a state-of-the-art survey. In: 2018 International Conference on Intelligent Systems (IS), pp. 698–705 (2018)

    Google Scholar 

  76. Verdecchia, R.: Identifying architectural technical debt in android applications through automated compliance checking. In: Proceedings of the 5th International Conference on Mobile Software Engineering and Systems, MOBILESoft 2018, pp. 35–36. ACM (2018)

    Google Scholar 

  77. Wheeler, D.A.: More than a gigabuck: estimating gnu/linux’s size (2001). https://dwheeler.com/sloc/redhat71-v1/redhat71sloc.html

  78. Zazworka, N., et al.: Comparing four approaches for technical debt identification. Software Qual. J. 22(3), 403–426 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Innopolis University, Innopolis, Russian Federation

    Ilya Khomyakov, Zufar Makhmutov, Ruzilya Mirgalimova & Alberto Sillitti

Authors
  1. Ilya Khomyakov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zufar Makhmutov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruzilya Mirgalimova
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alberto Sillitti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alberto Sillitti .

Editor information

Editors and Affiliations

  1. INSTICC and Polytechnic Institute of Setúbal, Setúbal, Portugal

    Joaquim Filipe

  2. Warsaw University of Technology, Warsaw, Poland

    Michał Śmiałek

  3. George Mason University, Fairfax, VA, USA

    Alexander Brodsky

  4. MODESTE/ESEO, Angers, France

    Slimane Hammoudi

Appendix

Appendix

Table 5. Techniques with input, output, and calculation.
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khomyakov, I., Makhmutov, Z., Mirgalimova, R., Sillitti, A. (2020). An Analysis of Automated Technical Debt Measurement. In: Filipe, J., Śmiałek, M., Brodsky, A., Hammoudi, S. (eds) Enterprise Information Systems. ICEIS 2019. Lecture Notes in Business Information Processing, vol 378. Springer, Cham. https://doi.org/10.1007/978-3-030-40783-4_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-40783-4_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-40782-7

  • Online ISBN: 978-3-030-40783-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation