Early evaluation of technical debt impact on maintainability
Introduction
Since Technical Debt was firstly introduced in Cunningham (1992), many approaches have emerged to identify (Vetro’ et al., 2010; Wong et al., 2011; Schumacher et al., 2010), estimate (Chin et al., 2010, Curtis et al., 2012a; Letouzey and Ilkiewicz, 2012, Marinescu, 2012) or, in general, deal with Technical Debt by different techniques (Ramasubbu and Kemerer, 2014). As the authors state in Kruchten et al. (2012), “most authors agree that the major cause of Technical Debt is schedule pressure, e.g. ignoring refactorings to reduce time to market” (Abad and Ruhe, 2015). However, as they also claim, Technical Debt is also related to quality problems being often produced by carelessness, lack of education, poor processes, lack of verification or, even, basic incompetence. These origins of Technical Debt are called unintentional debt (Brown et al., 2010) and examples of these quality problems occasioned by Technical Debt are bad reusability and low understandability (Griffith et al., 2014), error-prone and higher number of defects (Zazworka et al., 2014), negative impact on robustness, performance, security and transferability (Curtis et al., 2012a, 2012b) or, especially, on maintainability issues like stability (Zazworka et al., 2014). A study conducted by Chen and Huang (2009) highlights that stability is one of the top 10 higher-severity software development problem factors which affect software maintainability. Moreover, maintainability is currently draining 60–90% of the total cost of software development (Chen and Huang, 2009; Erlikh, 2000; Hung, 2007).
To solve these issues, several techniques have been proposed in the literature to detect Technical Debt in source code, such as the identification of modularity violations (Wong et al., 2011), code smells (Schumacher et al., 2010; Marinescu, 2004), grime buildups (Gueheneuc and Albin-Amiot, 2001; Izurieta and Bieman, 2007) or the identification of violations of good programmer practices by using Automatic Static Analysis (ASA) approaches (Vetro’ et al., 2010). Indeed, the combination of these four different techniques has been empirically evaluated in Zazworka et al. (2014) to test which practices perform better under different conditions and how they could complement each other to estimate Technical Debt interests (quality harms). Technical debt interest may be defined as the payment in the form of extra time, effort, and cost to address future changes in a project (Abad and Ruhe, 2015). Similarly, in Ramasubbu and Kemerer (2014), Griffith et al. (2014), Curtis et al. (2012b), and Zazworka et al. (2011), the authors conducted studies where they empirically evaluated the relation among different Technical Debt indicators and software quality characteristics in order to test whether the former are really related to the latter.
What all these works have in common is that they are focused on the programming level, when the system has already been implemented (if not completely, at least, partially). However, as claimed in Li et al. (2014), Technical Debt can span all the phases of the software lifecycle and there may also be sources of Technical Debt in non-code artifacts (Brown et al., 2010), e.g. requirements documents. Therefore, its identification at early stages of development may provide developers with important information to apply refactoring approaches (e.g. based on aspect-oriented techniques Moreira et al., 2013; Jacobson and Ng, 2004; Jacobson, 2003) improving, thus, modularity also at source code and therefore reducing Technical Debt at latest development stages (or, at least, reducing the future global interest). The reality is that requirements always change and Technical Debt is inevitable (Allman, 2012), however, the issue is not eliminating debt, but rather reducing it or even moving its identification to previous stages. Indeed, this is more important if we consider that those who incurred the debt may usually not be the same as those who will have to re-pay later (Brondum and Zhu, 2012).
Nevertheless, to the best of our knowledge, little effort has been dedicated to study the implications of Technical Debt at earlier stages of development. There are some works that have dealt with the definition of Technical Debt at the requirements level (Abad and Ruhe, 2015; Ernst, 2012) or its relation with architectural dependencies (Li et al., 2014; Brondum and Zhu, 2012). Even, these types of debts have been described in the mapping study introduced in Alves et al. (2016) as Requirements and Architecture Debts. However, the empirical evaluation of the quality problems produced by Technical Debt at early stages has been neglected in the literature so far. Based on this assumption, we have formulated the main question that we try to answer in this work: is there a relationship between Technical Debt indicators at the requirements level and software quality? Concretely, we focus on modularity violations (a well-known Technical Debt indicator Wong et al., 2011; Alves et al., 2016) and software stability (a quality attribute related to maintainability International Organization of Standardization, 2014). Thus, our main question is reformulated as follows: is there a relationship between modularity anomalies at the requirements level and system stability? The existence of this relationship would provide empirical evidence of the harmful relationship between Technical Debt and software quality at early stages of development.
To tackle the problem of answering this question, this paper presents an empirical study where we evaluate whether modularity anomalies at the requirements level occasioned by crosscutting concerns (Baniassad et al., 2006) are directly related to instability of the system, which would increase its interest. The empirical study is supported by the application of a conceptual framework defined in previous work (Conejero, 2010). The framework allows the identification of modularity violations based on scattering, tangling and crosscutting at any abstraction level but concretely at the requirements level. Moreover, based on this conceptual framework a set of software metrics were defined to quantify the Degree of Crosscutting properties that a system may have. In this work, these metrics are validated by comparing them with similar metrics introduced by other authors, whilst their utility is illustrated by comparing them with a set of metrics that measure stability. All the metrics are applied to measure both modularity and stability properties in three different software product lines (with different releases) and the measurements obtained are pairwise compared to test whether those metrics are correlated and to find an answer for our main question.
The rest of the paper is organized as follows. Section 2 briefly introduces the conceptual framework that supports the study by providing a method to identify crosscutting properties at requirements level. Section 3 presents the settings for our empirical study by introducing the hypothesis established, the measures used and the systems considered. Section 4 shows the results obtained and it discusses their interpretation according to our main hypothesis. Section 5 presents an evaluation of the metrics in order to select the most representative for future studies. Section 6 presents the threats to validity for this study. Finally, Section 7 discusses the related work and Section 8 concludes the paper.
Section snippets
Background
A concern is an interest, which pertains to the system's development, its operation or any other matters that are critical or otherwise important to one or more stakeholders (van den Berg et al., 2005). The term concern is closely related to the term feature (used in the Software Product Line context) in the sense of being a prominent or distinctive user-visible aspect, quality, or characteristic of a software system or systems (Kang et al., 1990). Software modularity is mainly determined by
Experimental design
As it has been commented in previous sections, the presence of crosscutting in a software system negatively affects its modularity and it is one of the most significant indicators of Technical Debt (Wong et al., 2011; Alves et al., 2016). However, modularity anomalies in a system may impact its quality in different ways, since other quality attributes could be affected, and it may increase interest in different ways. This work focuses on empirically evaluating whether modularity anomalies at
Results and discussion
This section presents the process followed to test the main hypothesis established in Section 3. This process is driven by the evaluation of the metrics introduced in the previous section and the analysis of their correlations. In other words, modularity metrics are empirically compared with stability ones to test their correlations. The process is composed by the next four different sub-steps (see Fig. 6): (1) the measurements for all the metrics in the different releases for each case study
Metrics evaluation
As it was mentioned in previous sections, the results obtained in the analysis presented may be also useful for further studies. Hence, with the aim of validating our metrics, in this section we pairwise compare the correlations obtained for all the modularity metrics to check whether the results obtained by our metrics (Table 3) are consistent with those obtained for the metrics previously introduced by other authors (Table 4). Moreover, based on the correlation coefficients obtained for
Threats to validity
In this section, we elaborate on several factors that may jeopardize the validity of our results. In order to present these threats, we follow the well-known categorization introduced by Wohlin et al. (2000), where threats are classified into four different validity categories: construct, internal, external and conclusion.
Related work
This section has been organized according to the different kind of works that are included. Firstly, we mention some works that focus on the identification of Technical Debt. Secondly, some works that empirically demonstrate the relationship between Technical Debt and software quality are presented. Finally, some works that deal with Technical Debt in early stages of development (Requirements or Architecture) are commented.
Conclusions
This paper has presented an empirical study where we analyzed the relation between Technical Debt, caused by modularity anomalies, and other software quality properties, namely maintainability attributes. The study has provided evidences of correlations among concern properties, such as scattering, tangling and crosscutting, and stability, one of the most important maintainability characteristics. Based on these correlations we extracted important conclusions that allow us to empirically prove
Acknowledgments
The authors gratefully acknowledge the support of TIN2015-69957-R (MINECO/FEDER, UE) project, Consejería de Economía e Infraestructuras/Junta de Extremadura (Spain)- European Regional Development Fund(ERDF)- GR15098 project and IB16055 project to the work presented here. This work was also partially supported by the 4IE project (0045-4IE-4-P) funded by the Interreg V-A España-Portugal (POCTEP) 2014-2020 program. We would like to thank A. Garcia and E. Figueiredo for allowing us to use the
José María Conejero received his Ph.D. in Computer Science from Universidad de Extremadura in 2010. He is an Assistant Professor at Universidad de Extremadura. He is the author of more than 20 papers of journals and conference proceedings and has also participated in different journals and conferences as member of the program committee. His research areas include the Aspect-Oriented Software Development, Requirements Engineering, Model-Driven Development or Ambient Intelligence.
References (65)
- et al.
Identification and management of technical debt: a systematic mapping study
Inf. Softw. Technol.
(2016) - et al.
An empirical analysis of the impact of software development problem factors on software maintainability
J. Syst. Softw.
(2009) - et al.
A systematic mapping study on technical debt and its management
J. Syst. Softw.
(2015) - et al.
An empirical investigation of modularity metrics for indicating architectural technical debt
- et al.
Using real options to manage technical debt in requirements engienering
- et al.
Principal component analysis
Wiley Interdiscip. Rev. Comput. Stat.
(2010) A model-driven approach for software product lines requirements engineering
Managing Technical Debt: Shortcuts that save money and time today can cost you down the road
ACM Queue
(2012)- et al.
Discovering early aspects
IEEE Softw.
(2006) - Briand, L., Morasca, S., Basili, V.R. Defining and Validating High-Level Design Metrics, University of Maryland at...
Visualising architectural dependencies
Managing technical debt in software-reliant systems
The economics of technical debt
Cut. IT J.
The Crosscutting PatternA Conceptual Framework for the Analysis of Modularity Across Software Development Phases
Early analysis of modularity in software product lines
The WyCash portfolio management system
Estimating the principal of an application's technical debt
IEEE Softw.
Estimating the size, cost, and types of technical debt
A Discipline of Programming
Distribution map
Do crosscutting concerns cause defects?
IEEE Trans. Softw. Eng.
The PLUSS approach – domain modeling with features, use cases and use case realizations
Leveraging legacy system dollars for e-business
IEEE IT Prof.
On the role of requirements in understanding and managing technical debt
Evolving software product lines with aspects
Technical debt indexes provided by tools: a preliminary discussion
Arcan: a tool for architectural smells detection
On the impact of aspectual decompositions on design stability: an empirical study
The correspondence between software quality models and technical debt estimation approaches
Integrating feature modeling with the RSEB
Using design patterns and constraints to automate the detection and correction of inter-class design defects
Cited by (13)
Exploring the Relation between Technical Debt Principal and Interest: An Empirical Approach
2020, Information and Software TechnologyCitation Excerpt :More specifically, as maintenance effort we assume the average lines of code added/deleted/modified between all pairs of successive versions of a system. This strategy has been used in a variety of studies e.g., [10,16,18,23,28], and [41]. Case study is an observational method that is used for studying phenomena in a real-life context [34].
Identifying the severity of technical debt issues based on semantic and structural information
2023, Software Quality JournalAutomated Identification and Prioritization of Self-Admitted Technical Debt Using NLP Word Embeddings
2023, International Conference on Self Sustainable Artificial Intelligence Systems, ICSSAS 2023 - ProceedingsCommunity Smell Occurrence Prediction on Multi-Granularity by Developer-Oriented Features and Process Metrics
2022, Journal of Computer Science and TechnologyThe Negative Implications of Technical Debt on Software Startups: What they are and when they surface
2022, Proceedings - 5th International Workshop on Software-Intensive Business: Towards Sustainable Software Business, IWSiB 2022
José María Conejero received his Ph.D. in Computer Science from Universidad de Extremadura in 2010. He is an Assistant Professor at Universidad de Extremadura. He is the author of more than 20 papers of journals and conference proceedings and has also participated in different journals and conferences as member of the program committee. His research areas include the Aspect-Oriented Software Development, Requirements Engineering, Model-Driven Development or Ambient Intelligence.
Roberto Rodríguez-Echeverría received his Ph.D. in Computer Science from Universidad de Extremadura in 2014. He is an Assistant Professor at Universidad de Extremadura. He is the author of more than 20 papers of journals and conference proceedings and has also participated in different journals and conferences as member of the program committee. His research areas include Web Engineering, Model-Driven Software Engineering, Software Modernization and End-User Development.
Juan Hernández received the B.Sc. in Mathematics from the University of Extremadura and the Ph.D. degree in Computer Science from the Technical University of Madrid. He is a Full Professor of the Quercus Software Engineering Group of the Extremadura University (Spain). His research interests include serviceoriented computing, ambient intelligence, aspect orientation and model driven development. He is involved in several research projects as responsible and senior researcher related to these subjects. He has published the results of his research in more than 100 papers in international journals, conference proceedings and book chapters. He has participated in many workshops and conferences as speaker and member of the program committee. He is currently member of the Spanish steering committee on Software Engineering and IEEE, and organized several workshops and international conferences.
Pedro J. Clemente is an Associate Professor of the Computer Science Department at the University of Extremadura (Spain). He received his BSc in Computer Science from the University of Extremadura in 1998 and a PhD in Computer Science in 2007. He has published numerous peer-reviewed papers in international journals, workshops, and conferences. His research interests include component-based software development, aspect orientation, service-oriented architectures, business process modeling, and model-driven development. He is involved in several research projects. He has participated in many workshops and conferences as speaker and member of the program committees.
Carmen Ortiz-Caraballo has a Ph.D. in Mathematics from the University of Seville (2011). She is an assistant professor of Mathematics at the Escola d'Enginyeria d'Igualada of the Universitat Politècnica de Catalunya (Spain). She has published several peer-reviewed papers in international journals, workshops, and conferences on harmonic analysis. She is involved in diferent research projects. Her research interests include harmonic analysis and applied mathematics, in which she is currently collaborating with different research groups.
Elena Jurado received the B.Sc. in Mathematics and the Ph.D. degree in Computer Science from the University of Extremadura (Spain) in 1985 and 2003, respectively. She has been a professor at University of Extremadura since 1985, and she is currently an associated professor. Her research interests include ambient intelligence, multidimensional indexing and content based information retrieval. She has published the results of his research in more than 20 papers in international journals and conference proceedings.
Fernando Sánchez-Figueroa received the Ph.D. degree from University of Extremadura. He is currently Professor at the Department of Computer Science, University of Extremadura, Spain. He belongs to the Quercus Software Engineering group, being his research focused on Web engineering, Big Data Visualization and Model Driven Engineering. He is also the CEO of Homeria Open Solutions, a spin-off arisen from Universidad de Extremadura.