Abstract
Software maintainability is often estimated by software developers and experts at the design phase to reduce maintenance efforts of software products. It is believed that the maintainability metrics help in throwing some light on the maintenance efforts of a software system. This paper studies the effectiveness of metrics of two maintainability components, namely modifiability and understandability in assessing maintenance efforts at the early design phase. To perform the empirical study, first of all, modifiability and understandability are estimated using certain design level metrics. A dataset of 776 class hierarchies from 20 different Java open source projects is considered with 12 different design level metrics as the independent variables and modifiability and understandability metrics as the dependent variables. Homogeneous as well as heterogeneous ensembles of MLR, MLP, ANN, and M5P methods are employed to predict maintainability of class hierarchies using design level metric dataset and the metric dataset of the two maintainability metrics. The concept of ROC curve for regression, ROC space and AOC introduced recently by Hernández-Orallo (Pattern Recogn 46:3395–3411, 2013) is used to determine the under/overestimation of individual models and their ensembles. It is concluded that most of the methods underestimate software maintainability that may lead to higher maintenance cost at a later stage and may also adversely affect the reputation of software companies. While modifiability is mostly underestimated, nothing could be concluded for understandability. The study is validated with the help of two available real datasets QUES and UIMS containing records of actual maintenance efforts and design level metrics dataset. These studies suggest that the definition of software modifiability metric needs a review.
Similar content being viewed by others
References
Al Dallal J (2011) Measuring the discriminative power of object-oriented class cohesion metrics. IEEE Trans Software Eng 37:788–804
Al Dallal J (2013) Object-oriented class maintainability prediction using internal quality attributes. Inf Softw Technol 55:2028–2048
Basili VR, Briand LC, Melo WL (1996) A validation of object-oriented design metrics as quality indicators. IEEE Trans Software Eng 22:751–761
Baudry B, Le Traon Y, Sunyé G (2002) Testability analysis of a UML class diagram. In: Proceedings of eighth IEEE symposium on software metrics. IEEE, pp 54–63
Bengtsson P, Bosch J (1999) Architecture level prediction of software maintenance. In: Proceedings of the third European conference on software maintenance and reengineering (Cat. No. PR00090). IEEE, pp 139–147
Bona GD, Forcina A, Falcone D (2018a) Maintenance strategy design in a sintering plant based on a multicriteria approach. International Journal of Management and Decision Making 17:29–49
Bona GD, Silvestri A, Forcina A, Petrillo A (2018b) Total efficient risk priority number (TERPN): a new method for risk assessment. J Risk Res 21:1384–1408
Briand LC, Wüst J, Daly JW, Porter DV (2000) Exploring the relationships between design measures and software quality in object-oriented systems. J Syst Softw 51:245–273
Bruntink M, van Deursen A (2006) An empirical study into class testability. J Syst Softw 79:1219–1232
Cartwright M, Shepperd M (2000) An empirical investigation of an object-oriented software system. IEEE Trans Software Eng 26:786–796
Chen J-C, Huang S-J (2009) An empirical analysis of the impact of software development problem factors on software maintainability. J Syst Softw 82:981–992
Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. IEEE Trans Software Eng 20:476–493
Daly J, Brooks A, Miller J, Roper M, Wood M (1996) Evaluating inheritance depth on the maintainability of objectoriented software. Empir Softw Eng 1:109–132
El Emam K, Melo W, Machado JC (2001) The prediction of faulty classes using object-oriented design metrics. J Syst Softw 56:63–75
Elish MO, Aljamaan H, Ahmad I (2015) Three empirical studies on predicting software maintainability using ensemble methods. Soft Comput 19:2511–2524
Genero M, Manso E, Visaggio A, Canfora G, Piattini M (2007) Building measure-based prediction models for UML class diagram maintainability. Empir Softw Eng 12:517–549
Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36
Harrison R, Counsell SJ, Nithi RV (1998) An evaluation of the MOOD set of object-oriented software metrics. IEEE Trans Softw Eng 24:491–496
Harrison R, Counsell S, Nithi R (2000) Experimental assessment of the effect of inheritance on the maintainability of object-oriented systems. J Syst Softw 52:173–179
Hernández-Orallo J (2013) ROC curves for regression. Pattern Recogn 46:3395–3411
Kiewkanya M, Jindasawat N, Muenchaisri P (2004) A methodology for constructing maintainability model of objectoriented design. In: Proceedings of fourth international conference on quality software, 2004. QSIC 2004, 2004. IEEE, pp 206–213
Kiran NR, Ravi V (2008) Software reliability prediction by soft computing techniques. J Syst Softw 81:576–583
Krogh A, Vedelsby J (1995) Neural network ensembles, cross validation, and active learning. Adv Neural Inf Process Syst 7:231–238
Laradji IH, Alshayeb M, Ghouti L (2015) Software defect prediction using ensemble learning on selected features. Inf Softw Technol 58:388–402
Li W, Henry S (1993) Object-oriented metrics that predict maintainability. J Syst Softw 23:111–122
Lincke R, Gutzmann T, Löwe W (2010) Software quality prediction models compared. In: 10th International conference on quality software (QSIC) 2010. IEEE, pp 82–91
Lu Y, Mao X, Li Z (2016) Assessing software maintainability based on class diagram design: a preliminary case study. Lect Notes Softw Eng 4:53
Mishra D (2012) New inheritance complexity metrics for object-oriented software systems: an evaluation with Weyuker’s properties. Comput Inform 30:267–293
Opitz D, Maclin R (1999) Popular ensemble methods: an empirical study. J Artif Intell Res 11:169–198
Orenyi BA, Basri S, Jung LT (2012) Object-oriented software maintainability measurement in the past decade. In: International conference on advanced computer science applications and technologies (ACSAT), IEEE, pp 257–262
Rathore SS, Kumar S (2016) A decision tree regression based approach for the number of software faults prediction. ACM SIGSOFT Softw Eng Notes 41:1–6
Rathore SS, Kumar S (2017) Towards an ensemble based system for predicting the number of software faults. Expert Syst Appl 82:357–382
Reddy BR, Ojha A (2014) Discrimination of class inheritance hierarchies–a vector approach. New Perspect Inf Syst Technol 2:121–130
Riaz M, Mendes E, Tempero E (2009) A systematic review of software maintainability prediction and metrics. In: Proceedings of the 2009 3rd international symposium on empirical software engineering and measurement, 2009. IEEE Computer Society, pp 367–377
Sheldon FT, Jerath K, Chung H (2002) Metrics for maintainability of class inheritance hierarchies. J Softw Maint Evol Res Pract 14:147–160
Subramanyam R, Krishnan MS (2003) Empirical analysis of ck metrics for object-oriented design complexity: implications for software defects. IEEE Trans Softw Eng 29:297–310
Thwin MMT, Quah T-S (2005) Application of neural networks for software quality prediction using object-oriented metrics. J Syst Softw 76:147–156
Ye F, Zhu X, Wang Y (2013) A new software maintainability evaluation model based on multiple classifiers combination. In: International conference on quality, reliability, risk, maintenance, and safety engineering (QR2MSE), IEEE, pp 1588–1591
Zhou Y, Leung H (2007) Predicting object-oriented software maintainability using multivariate adaptive regression splines. J Syst Softw 80:1349–1361
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Reddy, B.R., Ojha, A. How effective are maintainability metrics in estimating maintenance efforts? An empirical study. Int J Syst Assur Eng Manag 10, 984–1001 (2019). https://doi.org/10.1007/s13198-019-00828-3
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13198-019-00828-3