Abstract
Software maintainability prediction in the earlier stages of software development involves the construction of models for the accurate estimation of maintenance effort. This guides the software practitioners to manage the resources optimally. This study aims at systematically reviewing the prediction models from January 1990 to October 2019 for predicting software maintainability. We analyze the effectiveness of these models according to various aspects. To meet the goal of the research, we have identified 36 research papers. On investigating these papers, we found that various machine learning (ML), statistical (ST), and hybridized (HB) techniques have been applied to develop prediction models to predict software maintainability. The significant finding of this review is that the overall performance of ML-based models is better than that of ST models. The use of HB techniques for prediction of software maintainability is limited. The results of this review revealed that software maintainability prediction (SMP) models developed using ML techniques outperformed models developed using ST techniques. Also, the prediction performance of few models developed using HB techniques is encouraging, yet no conclusive results about the performance of HB techniques could be reported because different HB techniques are applied in a few studies.
Similar content being viewed by others
References
Aggarwal KK, Singh Y, Chhabra JK (2002) An integrated measure of software maintainability. In: Proceedings of the annual reliability and maintainability symposium, 2002, pp 235–241
Aggarwal KK, Singh Y, Chandra P, Puri M, Base R (2005) Measurement of software maintainability using a fuzzy model. J Comput Sci 1(4):538–542
Aggarwal KK, Singh Y, Kaur A, Malhotra R (2006) Application of artificial neural network for predicting maintainability using object- oriented metrics. Trans Eng Comput Technol 15:285–289
Al Dallal J (2013) Object-oriented class maintainability prediction using internal quality attributes. Inf Softw Technol 55(11):2028–2048
Al-Jamimi HA, Ahmed MA (2013) Machine learning approaches for predicting software maintainability: a fuzzy-based transparent model. IET Softw 7(6):317–326
Almugrin S, Albattah W, Melton A (2016) The journal of systems and software using indirect coupling metrics to predict package maintainability and testability. J Syst Softw 121:298–310
Anwar S, Ramzan M, Rauf A, Shahid AA (2010) Software maintenance prediction using weighted scenarios: an architecture perspective. International Conf Inf Sci Appl, 1–9
Arisholm E (2002) Dynamic coupling measures for object-oriented software. Proc Int Softw Metrics Symp 20:33–42
Bakar AD, Sultan AB, Zulzalil H, Din J (2012) Applying evolution programming search based software engineering (SBSE) in selecting the best open source software maintainability metrics. In: IEEE Symposium on Computer Applications and Industrial Electronics (ISCAIE), pp 70–73
Bakota T, Hegedus P, Ladanyi G, Kortvelyesi P, Ferenc R, Gyimothy T (2012) A cost model based on software maintainability. In: 28th IEEE international conference on software maintenance, pp 316–325
Balogh G, G, Végh ÁZ, Beszédes A (2012) Prediction of software development modification effort enhanced by a genetic algorithm, SSBSE Fast Abstract track, pp 1–6
Bandi RK, Vaishnavi VK, Turk DE (2003) Predicting maintenance performance using object-oriented design complexity metrics. IEEE Trans Softw Eng 29(1):77–87
Baqais AAB, Alshayeb M, Baig ZA (2013) Hybrid intelligent model for software maintenance prediction. Lecture Notes Eng Comput Sci 1:358–362
Boehm BW (1976) Software engineering. IEEE Trans Comput 25(12):1226–1241
Basgalupp MP, Barros RC, Ruiz DD, Carlos S, Brazil SP (2012) Predicting software maintenance effort through evolutionary-based decision trees. In: Proceedings of the 27th Annual ACM Symposium on Applied Computing, pp 1209–1214
Basili V, Briand L, Condon S (1996) Understanding and predicting the process of software maintenance releases. In: Proceedings—international conference on software engineering, pp 464–474
Bengtsson PO, Bosch J (1999) Architecture level prediction of software maintenance. In: Proceedings of the 3rd European conference on software maintenance and reengineering, pp 139–147
Bennett KH, Rajlich VT (2000) Software maintenance and evolution: a roadmap. Proc Conf Future Softw Eng 225:73–87
Bhutani S, Chug A (2015) Prediction of software maintainability using neural networks. Int J Comput Sci Commun Netw 5(2):92–95
Briand L, Daly W, Wust J (1999) A unified framework for coupling measurement in object-oriented systems. IEEE Trans Softw Eng 25(1):91–121
Broy M, Deissenboeck F, Pizka M (2006) Demystifying maintainability. In: Proceedings of the 2006 international workshop on Software quality—WoSQ ’06, pp 21–26
Campbell DT, Stanley JC, Gage NL (1963) Experimental and quasi-experimental designs for research. Houghton Mifflin, Boston
Chandra D (2012) Support vector approach by using radial kernel function for prediction of software maintenance effort on the basis of multivariate approach. Int J Comput Appl 51(4):21–25
Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. IEEE Trans Softw Eng 20:476–493
Chug A, Malhotra R (2016) Benchmarking framework for maintainability. Int J Innovat Comput Inf Control 12(2):615–634
Coleman D, Ash D, Lowther B, Oman P (1994) Using metrics to evaluate software system maintainability. Computer 27(8):44–49
Conte SD, Dunsmore HE, Shen VY (1986) Software engineering metrics and models. Benjamin-Cummings Publishing Co., Redwood City
Dagpinar M, Jahnke JH, Canada BC (2003) Predicting Maintainability with object-oriented metrics—an empirical comparison. In: Proceedings of the 10th working conference on reverse engineering, WCRE 2003, pp 155–164
Daly J, Brooks A, Miller J, Roper M, Wood M (1995) The effect of inheritance on the maintainability of object-oriented software: an empirical study. Proc Int Conf Softw Maint 44:20–29
De Lucia A, Pompella E, Stefanucci S (2005) Assessing effort estimation models for corrective maintenance through empirical studies. Inf Softw Technol 47(1):3–15
De Lucia A, Pompella E, Stefanucci S (2002) Effort estimation for corrective software maintenance. In: Proceedings of the 14th international conference on Software engineering and Knowledge Engineering, pp 409–416
Deligiannis I, Shepperd M (2004) A controlled experiment investigation of an object-oriented design heuristic for maintainability. J Syst Softw 72(2):129–143
Deligiannis I, Shepperd M, Roumeliotis M, Stamelos I (2003) An empirical investigation of an object-oriented design heuristic for maintainability. J Syst Softw 65(2):127–139
Elish MO, Elish KO (2009) Application of TreeNet in predicting object-oriented software maintainability: a comparative study. In: Proceedings of the european conference on software maintenance and reengineering, CSMR 2009, pp 69–78
Elish MO, Aljamaan H (2015) Three empirical studies on predicting software maintainability using ensemble methods. Soft Comput 19(19):2511–2524
Ferneley EH (1999) Design metrics as an aid to software maintenance: an empirical study. J Softw Evolut Process 11(1):55–72
Fioravanti F, Nesi P (2001) Estimation and prediction metrics for adaptive maintenance effort of object-oriented systems. IEEE Trans Softw Eng 27(12):1062–1083
Genero M, Genero M, Cantone G, Informatica D (2003) Building UML class diagram maintainability prediction models based on early metrics. In: Proceedings of 9th international software metrics symposium, pp 263–275
Haiquan Y, Gaoliang P, Wenjian L (2009) An application of case based reasoning to predict structure maintainability. Int Conf Comput Intell Softw Eng, pp 1–5
Halstead M (1977) Elements of software science. Elsevier, New York
Harrison R, Counsell S, Nithi R (2000) Experimental assessment of the effect of inheritance on the maintainability of object-oriented systems. J Syst Softw 52(2):173–179
Harrison R, Counsell S (1998) The role of inheritance in the maintainability of object-oriented systems. In: Proceedings of ESCOM, pp 449–458
Hatton L (2007) How accurately do engineers predict software maintenance tasks? Computer 40(2):64–69
Hayes JH (2004) A metrics-based software maintenance effort model. In: Proceedings of 8th Europian conference on software maintenance and re-engineering, pp 254–258
Hayes JH, Zhao L (2005) Maintainability prediction: a regression analysis of measures of evolving systems. In: 21st IEEE international conference on software maintenance (ICSM’05), pp 601–604
Hegedus P (2013) Revealing the effect of coding practices on software maintainability. In: IEEE International Conference on Software Maintenance, pp 578–581
Heitlager I, Kuipers T, Visser J (2007) A practical model for measuring maintainability. In: 6th International conference on the on quality of information and communications technology, pp 30–39
Henderson-Sellors B (1996) Software metrics. Prentice Hall, Upper Saddle River
Hirota T, Tohki M, Overstreet CM, Hashimoto M, Cherinka R (1994) An approach to predict software maintenance cost based on ripple complexity. In: Proceedings of 1st asia-pacific software engineering conference, pp 439–444
Hops JM, Sherif JS (1995) Development and application of composite complexity models and a relative complexity metric in a software maintenance environment. J Syst Softw 31(2):157–169
IEEE: IEEE Standard Glossary of Software Engineering Terminology. IEEE Press, Piscataway (1990)
Jain R, Sharma A (2016) Critical insight into maintainability and reliability assessment aspects of software systems using soft computing approaches. Int J Control Theory Appl 9(20):9397–9404
Jain A, Tarwani S, Chug A (2016) An empirical investigation of evolutionary algorithm for software maintainability prediction. In: 2016 IEEE Students’ conference on electrical, electronics and computer science, SCEECS-2016, pp 1–6
Jeet K, Dhir R, Verma H (2012) A comparative study of bayesian and fuzzy approach to assess and predict maintainability of the software using activity-based quality model. SIGSOFT Softw Eng Notes 37(3):1–9
Jeet K, Dhir R (2012) Bayesian and fuzzy approach to assess and predict the maintainability of software: a comparative study. ISRN Software Engineering, pp 1–10
Jin C, Jin SW (2015) Prediction approach of software fault-proneness based on hybrid artificial neural network on quantum particle swarm optimization. Appl Soft Comput 35:717–725
Jindal R, Malhotra R, Jain A (2015) Predicting software maintenance effort using neural networks. In: 2015 4th International Conference on Reliability, Infocom Technologies and Optimization: Trends and Future Directions, pp 1–6
Jin C, Liu JA (2010) Applications of support vector mathine and unsupervised learning for predicting maintainability using object-oriented metrics. In: International conference on multimedia and information technology, MMIT 2010 vol 1, pp 24–27
Kaur A, Kaur K (2013) Statistical comparison of modelling methods for software maintainability prediction. Int J Softw Eng Knowl Eng 23(6):743–747
Kaur K, Singh H (2011) Determination of maintainability index for object oriented systems. ACM SIGSOFT Softw Eng Notes 36(2):1–6
Kaur A, Kaur K, Malhotra R (2010) Soft computing approaches for prediction of software maintenance effort. Int J Comput Appl 1(16):69–75
Kaur A, Kaur K, Pathak K (2014) Software maintainability prediction by data mining of software code metrics. In: 2014 International conference on data mining and intelligent computing, ICDMIC 2014, pp 1–6
Kewen L, Chenxi C, Wenying L, Xianghua F, Qinghua L (2014) Software defect prediction using fuzzy integral fusion based on GA-FM. Wuhan Univ J Natural Sci 19(5):405–408
Khoshgoftaar TM, Szabo RM (1994a) Improving code churn predictions during the system test and maintenance phases. In: Proceedings international conference on software maintenance, ICSM 1994 vol 7, pp 58–67
Khoshgoftaar TM, Szabo RM (1994b) Improving neural network predictions of software quality using principal components analysis. In: Proceedings of IEEE international conference on neural networks, ICNN 1994 vol 5, pp 3295–3300
Kitechenham BA (2007) Guidelines for performing systematic literature review in software engineering, Technical report EBSE-2007-001, UK
Ku Y, Du J, Yang Y, Wang Q (2011) Estimating software maintenance effort from use cases: an industrial case study. IEEE international conference on software maintenance, ICSM, pp 482–491
Kumar L (2016) Maintainability prediction of web service using support vector machine with various kernel methods. Int J Syst Assur Eng Manag 8(2):205–222
Kumar L, Rath SK (2016) Hybrid functional link artificial neural network approach for predicting maintainability of object-oriented software R. J Syst Softw 121:170–190
Kumar L, Rath SK (2017) Software maintainability prediction using hybrid neural network and fuzzy logic approach with parallel computing concept. Int J Syst Assur Eng Manag 8(2):1487–1502
Kumar V, Kumar R, Sharma A (2014) Maintainability prediction from project metrics data analysis using artificial neural network. Middle-East J Sci Res 19(10):1412–1420
Kumar L, Kumar D, Kumar S (2015) Validating the effectiveness of object-oriented metrics for predicting maintainability. Procedia Comput Sci 57:798–806
Kumar L, Lal S, Murthy LB (2019) Estimation of maintainability parameters for object-oriented software using hybrid neural network and class level metrics. Int J Syst Assur Eng Manag, 1–31
Kumar L, Rath S (2015) Neuro-Genetic Approach for predicting maintainability using Chidamber and Kemerer software metrics suite. Recent Advances in Information and Communication Technology, Springer International Publishing, pp 31–40
Kumar L, Rath SK (2015) Predicting object-oriented software maintainability using hybrid neural network with parallel computing concept. In: Proceedings of the 8th India software engineering conference, ISEC-2015, pp 100–109
Lagerström R, Johnson P (2008) Using architectural models to predict the maintainability of enterprise systems. In: 12th european conference on software maintenance and reengineering, IEEE, pp 248–252
Land R (2002) Measurements of Software Maintainability. In: proceedings of ARTES Graduate Student Conference
Li YF, Xie M, Goh TN (2008) A bayesian inference approach for probabilistic analogy based software maintenance effort estimation. In: Proceedings of the 14th IEEE pacific rim international symposium on dependable computing, PRDC 2008, pp 176–183
Li W, Henry S (1993) Object-oriented metrics that predict maintainability. J Syst Softw 23(2):111–122
Lorenz M, Kidd J (1994) Object-oriented software metrics. Prentice-Hall, Upper Saddle River
Lowther B, Oman P (1995) The application of software maintainability in industrial software systems. J Syst Softw 29(1):3–16
Malhotra R (2015) A systematic review of machine learning techniques for software fault prediction. Appl Soft Comput 27:504–518
Malhotra R, Chug A (2012) Software maintainability prediction using machine learning algorithms. Softw Eng nt J (SEIJ) 2(2):19–36
Malhotra R, Chug A (2014a) Application of Group Method of Data Handling model for software maintainability prediction using object oriented systems. IntJ Syst Assur Eng Manag 5(2):165–173
Malhotra R, Chug A (2014b) Application of evolutionary algorithms for software maintainability prediction using object-oriented metrics. In: Proceedings of the 8th international conference on bioinspired information and communications technologies, pp 348–351
Malhotra R, Chug A (2016) Software maintainability: systematic literature review and current trends. Int J Softw Eng Knowl Eng 26(8):1221–1253
Malhotra R, Lata K (2017) An exploratory study for predicting maintenance effort using hybridized techniques. In: Proceedings of the 10th innovations in software engineering conference. ACM, pp 26–33
Malhotra R, Lata K (2018) On the application of cross-project validation for predicting maintainability of open source software using machine learning techniques. In: 7th IEEE International conference on reliability, infocom technologies and optimization (ICRITO), pp 175–181
Manchanda S, Chug A (2015) CFS based feature subset selection for software maintainance prediction. Int J Adv Found Res Comput 2(5):13–23
Mie M, Thwin T, Quah T (2003) Application of neural networks for software quality prediction using object-oriented metrics. In: Proceedings of the international conference on software maintenance, ICSM 2003, vol 76, pp 116–125
Mishra S, Sharma A (2015) Maintainability prediction of object oriented software by using adaptive network based fuzzy system technique. Int J Comput Appl 119(9):24–27
Misra SC (2005) Modelling design/coding factors that drive maintainability of software systems. Softw Q J 13(3):297–320
Muthanna S, Ponnambalaml KKK, Stacey B (2000) A maintainability model for industrial software systems using design level metrics. In: Proceeding 7th working conference on reverse engineering, pp 248–256
Niessink F, Van Vliet H (1997)Predicting maintenance effort with function points. Int Conf Softw Maint 32–39
Olatunji SO (2013) Sensitivity based linear learning method and extreme learning machines compared for software maintainability prediction of object orinented software system. ICTACT J Soft Comput 3:514–523
Oman P, Hagemeister J (1992) Metrics for assessing a software system’s maintainability. In: Software Maintenance procedings, pp 337–334
Perepletchikov M (2007) Coupling metrics for predicting maintainability in service-oriented designs. In: 18th Australian conference on software engineering, pp 329–340
Ping L (2010) A quantitative approach to software maintainability prediction. In: Proceedings of 2010 international forum on information technology and applications, IFITA 2010, vol 1, pp 105–108
Prasanth NN, Ganesh S, Dalton GA (2008) “Prediction of maintainability using software complexity analysis: an extended FRT. In: Proceedings of the 2008 international conference on computing, communication and networking, ICCCN 2008, pp 1–9
Prechelt L, Unger B, Philippsen M, Tichy W (1998) Re-Evaluating inheritance depth on the maintainability of object-oriented software. Int J Empir Softw Eng pp 1–16
Rai V, Srivastava AM, Pandey H, Singh VK (2015) Estimation of maintainability in object oriented design phase: state of the art. Int J Sci Eng Res 6(9):25–35
Rana SKA, Dash Y (2012) Maintainability prediction of object-oriented software system by multilayer perceptron model. ACM SIGSOFT Softw Eng Notes 37(5):1–4
Riaz M, Mendes E, Tempero E (2009) A systematic review of software maintainability prediction and metrics. In: Proceedings of the 3rd international symposium on empirical software engineering and measurement. IEEE Computer Society, pp 367–377
Riaz M, Mendes E, Tempero E, Sulayman M (2013) Using CBR and CART to predict maintainability of relational database-driven software applications. In: Proceedings of the 17th international conference on evaluation and assessment in software engineering—EASE ’13, p 132
Rizvi SWA, Khan RA (2010) Maintainability estimation model for object-oriented software in design phase. J Comput 2(4):26–32
Schneberger SL (1997) Distributed computing environments: effects on software maintenance difficulty. J Syst Softw 37(2):101–116
Seref B, Tanriover O (2016) Softwarecode maintainability: a litreature review. Int J Softw Eng Appl (IJSEA) 7(3):69–87
Sharawat S (2012) Software maintainability prediction using neural networks. Int J Eng Res Appl (IJERA) 2(2):750–755
Sharma H, Chug A (2015) Dynamic metrics are superior than static metrics in maintainability prediction: an empirical case study. In: 4th International conference on reliability, infocom technologies and optimization, ICRITO-2015, pp 2–7
Sheshasaayee A, Jose MR (2016) Deployment of a fuzzy XNOR based evaluative system for the maintainability features of aspect oriented systems. Int J Appl Eng Res 11(9):6637–6642
Singh RK, Balyan A (2012) Approach to software maintainability prediction versus performance. Int J Soft Comput Eng 2(2):51–54
Singh Y, Bhatia PK, Sangwan O (2009) Predicting software maintenance using fuzzy model. ACM SIGSOFT Softw Eng Notes 34(4):1–6
Somerville I (2012) Software engineering. Addison-Wesley, Boston
Tarwani S, Chug A (2016) Predicting maintainability of open source software using gene expression programming and bad smells. In: 5th International conference on reliability, in infocom technologies and optimization (Trends and Future Directions), ICRITO-2016, pp 452–459
Van Koten C, Gray AR (2006) An application of Bayesian network for predicting object-oriented software maintainability. Inf Softw Technol 48(1):59–67
Wang LJ, Hu XX, Ning ZY, Ke WH (2009) Predicting object-oriented software maintainability using projection pursuit regression. In: 1st International conference on information science and engineering, ICISE 2009, pp 3827–3830
Wen J, Li S, Lin Z, Hu Y, Huang C (2012) Systematic literature review of machine learning based software development effort estimation models. Inf Softw Technol 54(1):41–59
Ye F, Zhu X, Wang Y (2013) A new software maintainability evaluation model based on multiple classifiers combination. IEEE International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering, pp 1588–1591
Zavvar M, Ramezani F (2015) Measuring of software maintainability using adaptive fuzzy neural network. Int J Modern Edu Comput Sci 7(10):27–32
Zhang W, Huang LG, Ng V, Ge J (2015) SMPLearner: learning to predict software maintainability. Autom Soft Eng 22(1):111–141
Zhou Y, Leung H (2007) Predicting object-oriented software maintainability using multivariate adaptive regression splines. J Syst Softw 80(8):1349–1361
Zhou Y, Xu B, Yuming Z, Baowen XU (2008) Predicting the maintainability of open source software using design metrics. Wuhan Univ J Natural Sci 13(1):14–20
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Authors certify that this manuscript has not been submitted to more than one journal for simultaneous consideration and it has not been published previously (partly or in full). Authors certify that no funding has been received for the conduct of this study and preparation of this manuscript. This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
1.1 6.1 Quality Assessment Questionnaire
All candidate studies selected for this systematic review were evaluated on the basis of quality questions stated in Table 14. This table describes the percentage of studies in which the quality questions were addressed completely, partially or not addressed.
1.2 6.2 Research questions addressed in selected primary studies
Table 15 describes the research questions that are addressed in all selected primary studies individually.
Rights and permissions
About this article
Cite this article
Malhotra, R., Lata, K. A systematic literature review on empirical studies towards prediction of software maintainability. Soft Comput 24, 16655–16677 (2020). https://doi.org/10.1007/s00500-020-05005-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-020-05005-4