Abstract
Background: Measuring and understanding Technical Debt (TD) is quite complex since there are a number of different definitions and techniques that have been proposed in the last few years and it is not clear which ones should be used in which conditions. The approaches proposed are almost never based on the existing ones and their validation is often performed in a very limited number of projects. For this reasons, practitioners are confused and find difficult to apply such approaches in their projects.
Goals: This paper investigates the available techniques for evaluating TD using automated tools aiming at helping practitioners and researcher in understanding the available options and apply them correctly.
Method: The study has been performed as a Systematic Literature Review (SLR) applied to 835 studies obtained from the three largest digital libraries and databases.
Results: After applying all filtering stages, 38 papers out of 835 have been selected and analyzed in depth. Almost all of them propose novel approaches to measure TD using different criteria and they do not extend or validate existing approaches.
Conclusions: The area is not mature and it lacks independent evaluations of the models proposed. Authors focus on proposing new approaches and no consolidation can be identified. Moreover, almost all the approaches proposed are automated only partially and through prototype tools designed just to support the studies analyzed in the paper in which the approach is proposed and rarely maintained. These facts makes difficult the application of such methods by practitioners.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Scientific toolworks, inc. understand 2.6. http://www.scitools.com/
de Almeida, R.R., Treude, C., Kulesza, U.: Tracy: a business-driven technical debt prioritization framework. In: 35th International Conference on Software Maintenance and Evolution (ICSME 2019) (2019)
Alves, N.S., Mendes, T.S., de Mendonça, M.G., SpÃnola, R.O., Shull, F., Seaman, C.: Identification and management of technical debt: a systematic mapping study. Inf. Softw. Technol. 70, 100–121 (2016)
Ampatzoglou, A., Ampatzoglou, A., Avgeriou, P., Chatzigeorgiou, A.: Establishing a framework for managing interest in technical debt. In: 5th International Symposium on Business Modeling and Software Design, BMSD (2015)
Ampatzoglou, A., Ampatzoglou, A., Chatzigeorgiou, A., Avgeriou, P.: The financial aspect of managing technical debt: a systematic literature review. Inf. Softw. Technol. 64, 52–73 (2015)
Arcelli-Fontana, F., Pigazzini, I., Roveda, R., Tamburri, D., Zanoni, M., Nitto, E.: ARCAN: a tool for architectural smells detection. In: Proceeding of the International Conference on Software Architecture, ICSA 2017, pp. 282–285. IEEE (2017)
Arvedahl, S.: Introducing debtgrep, a tool for fighting technical debt in base station software. In: Proceedings of the 2018 International Conference on Technical Debt TechDebt 2018, pp. 51–52. ACM (2018)
Ayewah, N., Hovemeyer, D., Morgenthaler, J.D., Penix, J., Pugh, W.: Using static analysis to find bugs. IEEE Softw. 25(5), 22–29 (2008)
Behutiye, W.N., RodrÃguez, P., Oivo, M., Tosun, A.: Analyzing the concept of technical debt in the context of agile software development: a systematic literature review. Inf. Softw. Technol. 82, 139–158 (2017)
BenIdris, M., Ammar, H., Dzielski, D.: Investigate, identify and estimate the technical debt: a systematic mapping study. Int. J. Softw. Eng. Appl. 9(5), 1–14 (2018)
Besker, T., Martini, A., Bosch, J.: A systematic literature review and a unified model of ATD. In: 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 189–197. IEEE (2016)
Besker, T., Martini, A., Bosch, J.: Managing architectural technical debt: a unified model and systematic literature review. J. Syst. Softw. 135, 1–16 (2018)
Biaggi, A., Arcelli Fontana, F., Roveda, R.: An architectural smells detection tool for C and C++ projects. In: 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 417–420 (2018)
Bougouffa, S., Dong, Q.H., Diehm, S., Gemein, F., Vogel-Heuser, B.: Technical debt indication in plc code for automated production systems: Introducing a domain specific static code analysis tool. IFAC-Papers OnLine 51(10), 70–75 (2018). 3rd IFAC Conference on Embedded Systems, Computational Intelligence and Telematics in Control CESCIT 2018
Britsman, E., Tanriverdi, Ö.: Identifying technical debt impact on maintenance effort-an industrial case study (2015). https://gupea.ub.gu.se/handle/2077/40110
Capitán, L., Vogel-Heuser, B.: Metrics for software quality in automated production systems as an indicator for technical debt. In: 2017 13th IEEE Conference on Automation Science and Engineering (CASE), pp. 709–716 (2017)
Capitán, L., Vogel-Heuser, B.: Metrics for software quality in automated production systems as an indicator for technical debt. In: 2017 13th IEEE Conference on Automation Science and Engineering (CASE), pp. 709–716, August 2017
Chatzigeorgiou, A., Ampatzoglou, A., Ampatzoglou, A., Amanatidis, T.: Estimating the breaking point for technical debt. In: 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD), pp. 53–56. IEEE (2015)
Ciolkowski, M., Guzmán, L., Trendowicz, A., Salfner, F.: Lessons learned from the ProDebt research project on planning technical debt strategically. In: Felderer, M., Méndez Fernández, D., Turhan, B., Kalinowski, M., Sarro, F., Winkler, D. (eds.) PROFES 2017. LNCS, vol. 10611, pp. 523–534. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69926-4_42
Coman, I.D., Sillitti, A., Succi, G.: Investigating the usefulness of pair-programming in a mature agile team. In: 9th International Conference on eXtreme Programming and Agile Processes in Software Engineering (XP2008), June 2008
Corral, L., Sillitti, A., Succi, G.: Software development processes for mobile systems: is agile really taking over the business? In: 1st International Workshop on Mobile-Enabled Systems (MOBS 2013) at ICSE 2013, June 2013
Cunningham, W.: The WyCash portfolio management system. In: Addendum to the Proceedings on Object-Oriented Programming Systems, Languages, and Applications (Addendum), OOPSLA 1992, Vancouver, British Columbia, Canada, pp. 29–30. Association for Computing Machinery, New York (1992). https://doi.org/10.1145/157709.157715
Curtis, B., Sappidi, J., Szynkarski, A.: Estimating the size, cost, and types of technical debt. In: Proceedings of the Third International Workshop on Managing Technical Debt, pp. 49–53. IEEE Press (2012)
Falessi, D., Reichel, A.: Towards an open-source tool for measuring and visualizing the interest of technical debt. In: 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD), pp. 1–8. IEEE (2015)
Falessi, D., Shaw, M.A., Shull, F., Mullen, K., Keymind, M.S.: Practical considerations, challenges, and requirements of tool-support for managing technical debt. In: 2013 4th International Workshop on Managing Technical Debt (MTD), pp. 16–19. IEEE (2013)
Flisar, J., Podgorelec, V.: Identification of self-admitted technical debt using enhanced feature selection based on word embedding. IEEE Access 7, 106475–106494 (2019)
Gaudin, O.: Evaluate your technical debt with sonar. Sonar, June 2009
Griffith, I., Reimanis, D., Izurieta, C., Codabux, Z., Deo, A., Williams, B.: The correspondence between software quality models and technical debt estimation approaches. In: 2014 Sixth International Workshop on Managing Technical Debt (MTD), pp. 19–26. IEEE (2014)
Gruber, H., Plösch, R., Saft, M.: On the validity of benchmarking for evaluating code quality. In: IWSM/MENSURA 2010 (2010). https://www.iwsm-mensura.org/2010-conference/
Heitlager, I., Kuipers, T., Visser, J.: A practical model for measuring maintainability. In: 6th International Conference on the Quality of Information and Communications Technology, QUATIC 2007, pp. 30–39. IEEE (2007)
Izurieta, C., Griffith, I., Reimanis, D., Luhr, R.: On the uncertainty of technical debt measurements. In: 2013 International Conference on Information Science and Applications (ICISA), pp. 1–4. IEEE (2013)
Kamei, Y., Maldonado, E.D.S., Shihab, E., Ubayashi, N.: Using analytics to quantify interest of self-admitted technical debt. In: QuASoQ/TDA@ APSEC, pp. 68–71 (2016)
Khomyakov, I., Makhmutov, Z., Mirgalimova, R., Sillitti, A.: Automated measurement of technical debt: a systematic literature review. In: 21st International Conference on Enterprise Information Systems (ICEIS 2019), May 2019
Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering (version 2.3). Technical report, Keele University and University of Durham (2007)
Kumar, S., Bahsoon, R., Chen, T., Buyya, R.: Identifying and estimating technical debt for service composition in SaaS cloud. In: 2019 IEEE International Conference on Web Services (ICWS), pp. 121–125, July 2019
Lavazza, L., Morasca, S., Tosi, D.: A method to optimize technical debt management in timed-boxed processes. In: The Thirteenth International Conference on Software Engineering Advances (ICSEA 2018) (2018)
Lenarduzzi, V., Sillitti, A., Taibi, D.: Analyzing forty years of software maintenance models. In: 39th International Conference on Software Engineering (ICSE 2017), May 2017
Lenarduzzi, V., Martini, A., Taibi, D., Tamburri, D.A.: Towards surgically-precise technical debt estimation: early results and research roadmap. In: Proceedings of the 3rd ACM SIGSOFT International Workshop on Machine Learning Techniques for Software Quality Evaluation, MaLTeSQuE 2019, pp. 37–42. ACM (2019)
Lenarduzzi, V., Saarimäki, N., Taibi, D.: The technical debt dataset. In: Proceedings of the Fifteenth International Conference on Predictive Models and Data Analytics in Software Engineering PROMISE 2019, pp. 2–11. ACM (2019)
Letouzey, J.L.: The SQALE method for evaluating technical debt. In: 2012 Third International Workshop on Managing Technical Debt (MTD), pp. 31–36. IEEE (2012)
Letouzey, J.L., Ilkiewicz, M.: Managing technical debt with the SQALE method. IEEE Softw. 29(6), 44–51 (2012)
Li, Z., Avgeriou, P., Liang, P.: A systematic mapping study on technical debt and its management. J. Syst. Softw. 101, 193–220 (2015)
Li, Z., Liang, P., Avgeriou, P., Guelfi, N., Ampatzoglou, A.: An empirical investigation of modularity metrics for indicating architectural technical debt. In: Proceedings of the 10th international ACM Sigsoft conference on Quality of Software Architectures, pp. 119–128. ACM (2014)
Luhr, R.L., et al.: The application of technical debt mitigation techniques to a multidisciplinary software project. Ph.D. thesis, Montana State University-Bozeman, College of Engineering (2015)
Maldonado, E.D.S., Shihab, E.: Detecting and quantifying different types of self-admitted technical debt. In: 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD), pp. 9–15. IEEE (2015)
Marinescu, R.: Assessing technical debt by identifying design flaws in software systems. IBM J. Res. Dev. 56(5), 9–1 (2012)
Martini, A., Bosch, J., Chaudron, M.: Architecture technical debt: understanding causes and a qualitative model. In: Proceedings of the 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications, SEAA 2014, pp. 85–92. IEEE (2014)
Martini, A., Sikander, E., Madlani, N.: A semi-automated framework for the identification and estimation of architectural technical debt: a comparative case-study on the modularization of a software component. Inf. Softw. Technol. 93, 264–279 (2018)
Mayr, A., Plösch, R., Körner, C.: A benchmarking-based model for technical debt calculation. In: 2014 14th International Conference on Quality Software (QSIC), pp. 305–314. IEEE (2014)
Mendes, T.S., Gomes, F.G.S., Gonçalves, D.P., Mendonça, M.G., Novais, R.L., SpÃnola, R.O.: VisminerTD: a tool for automatic identification and interactive monitoring of the evolution of technical debt items. J. Brazil. Comput. Soc. 25(1), 2 (2019)
Monteith, J.Y., McGregor, J.D.: Exploring software supply chains from a technical debt perspective. In: Proceedings of the 4th International Workshop on Managing Technical Debt, pp. 32–38. IEEE Press (2013)
Nugroho, A., Visser, J., Kuipers, T.: An empirical model of technical debt and interest. In: Proceedings of the 2nd Workshop on Managing Technical Debt, pp. 1–8. ACM (2011)
Pacheco, A., MarÃn-Raventós, G., López, G.: Designing a technical debt visualization tool to improve stakeholder communication in the decision-making process: a case study. In: Tjoa, A.M., Raffai, M., Doucek, P., Novak, N.M. (eds.) CONFENIS 2018. LNBIP, vol. 327, pp. 15–26. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99040-8_2
Parodi, E., Matalonga, S., Macchi, D., Solari, M.: Comparing technical debt in student exercises using test driven development, test last and ad hoc programming. In: 2016 XLII Latin American Computing Conference (CLEI), pp. 1–10. IEEE (2016)
Parthiban, D.G.: Examination of tools for managing different dimensions of technical debt. CoRR abs/1904.11062 (2019). http://arxiv.org/abs/1904.11062
Pecorelli, F., Di Nucci, D., De Roover, C., De Lucia, A.: On the role of data balancing for machine learning-based code smell detection. In: Proceedings of the 3rd ACM SIGSOFT International Workshop on Machine Learning Techniques for Software Quality Evaluation, MaLTeSQuE 2019, pp. 19–24. ACM (2019)
Ploesch, R., Gruber, H., Pomberger, G., Saft, M., Schiffer, S.: Tool support for expert-centred code assessments. In: 2008 1st International Conference on Software Testing, Verification, and Validation, pp. 258–267. IEEE (2008)
Poliakov, D., et al.: A systematic mapping study on technical debt definition (2015). https://pdfs.semanticscholar.org/a425/2d6a5d9522a85984379f8ee3bf118df782c1.pdf
Potdar, A., Shihab, E.: An exploratory study on self-admitted technical debt. In: 2014 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 91–100. IEEE (2014)
Ramasubbu, N., Kemerer, C.F.: Integrating technical debt management and software quality management processes: a normative framework and field tests. IEEE Trans. Software Eng. 45(3), 285–300 (2019)
Ren, X., Xing, Z., Xia, X., Lo, D., Wang, X., Grundy, J.: Neural network-based detection of self-admitted technical debt: from performance to explainability. ACM Trans. Softw. Eng. Methodol. 28(3), 15:1–15:45 (2019)
Ribeiro, L.F., de Freitas Farias, M.A., Mendonça, M.G., SpÃnola, R.O.: Decision criteria for the payment of technical debt in software projects: a systematic mapping study. In: ICEIS, no. 1, pp. 572–579 (2016)
Rios, N., de Mendonça Neto, M.G., SpÃnola, R.O.: A tertiary study on technical debt: types, management strategies, research trends, and base information for practitioners. Inf. Softw. Technol. 102, 117–145 (2018)
Sierra, G., Shihab, E., Kamei, Y.: A survey of self-admitted technical debt. J. Syst. Softw. 152, 70–82 (2019)
Singh, V., Pollock, L.L., Snipes, W., Kraft, N.A.: A case study of program comprehension effort and technical debt estimations. In: 2016 IEEE 24th International Conference on Program Comprehension (ICPC), pp. 1–9. IEEE (2016)
Singh, V., Snipes, W., Kraft, N.A.: A framework for estimating interest on technical debt by monitoring developer activity related to code comprehension. In: 2014 Sixth International Workshop on Managing Technical Debt (MTD), pp. 27–30. IEEE (2014)
Skiada, P., Ampatzoglou, A., Arvanitou, E., Chatzigeorgiou, A., Stamelos, I.: Exploring the relationship between software modularity and technical debt. In: 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 404–407 (2018)
Skourletopoulos, G., Mavromoustakis, C.X., Mastorakis, G., Rodrigues, J.J., Chatzimisios, P., Batalla, J.M.: A fluctuation-based modelling approach to quantification of the technical debt on mobile cloud-based service level. In: 2015 IEEE Globecom Workshops (GC Wkshps), pp. 1–6. IEEE (2015)
Snipes, W., Nair, A.R., Murphy-Hill, E.: Experiences gamifying developer adoption of practices and tools. In: Companion Proceedings of the 36th International Conference on Software Engineering, pp. 105–114. ACM (2014)
Soudris, D., et al.: Exa2pro programming environment: architecture and applications. In: Proceedings of the 18th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation, SAMOS 2018, pp. 202–209. ACM (2018)
SpÃnola, R.O., Zazworka, N., Vetro, A., Shull, F., Seaman, C.: Understanding automated and human-based technical debt identification approaches-a two-phase study. J. Brazil. Comput. Soc. 25(1), 5 (2019)
Tom, E., Aurum, A., Vidgen, R.: An exploration of technical debt. J. Syst. Softw. 86(6), 1498–1516 (2013)
Tsantalis, N., Chaikalis, T., Chatzigeorgiou, A.: Jdeodorant: identification and removal of type-checking bad smells. In: 12th European Conference on Software Maintenance and Reengineering, CSMR 2008, pp. 329–331. IEEE (2008)
Tsintzira, A.A., Ampatzoglou, A., Matei, O., Ampatzoglou, A., Chatzigeorgiou, A., Heb, R.: Technical debt quantification through metrics: an industrial validation. In: 15th China-Europe International Symposium on Software Engineering Education (CEISEE 2019). IEEE (2019)
Tsoukalas, D., Siavvas, M., Jankovic, M., Kehagias, D., Chatzigeorgiou, A., Tzovaras, D.: Methods and tools for td estimation and forecasting: a state-of-the-art survey. In: 2018 International Conference on Intelligent Systems (IS), pp. 698–705 (2018)
Verdecchia, R.: Identifying architectural technical debt in android applications through automated compliance checking. In: Proceedings of the 5th International Conference on Mobile Software Engineering and Systems, MOBILESoft 2018, pp. 35–36. ACM (2018)
Wheeler, D.A.: More than a gigabuck: estimating gnu/linux’s size (2001). https://dwheeler.com/sloc/redhat71-v1/redhat71sloc.html
Zazworka, N., et al.: Comparing four approaches for technical debt identification. Software Qual. J. 22(3), 403–426 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Khomyakov, I., Makhmutov, Z., Mirgalimova, R., Sillitti, A. (2020). An Analysis of Automated Technical Debt Measurement. In: Filipe, J., Śmiałek, M., Brodsky, A., Hammoudi, S. (eds) Enterprise Information Systems. ICEIS 2019. Lecture Notes in Business Information Processing, vol 378. Springer, Cham. https://doi.org/10.1007/978-3-030-40783-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-40783-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-40782-7
Online ISBN: 978-3-030-40783-4
eBook Packages: Computer ScienceComputer Science (R0)