Skip to main content
Log in

A systematic literature review on empirical studies towards prediction of software maintainability

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Software maintainability prediction in the earlier stages of software development involves the construction of models for the accurate estimation of maintenance effort. This guides the software practitioners to manage the resources optimally. This study aims at systematically reviewing the prediction models from January 1990 to October 2019 for predicting software maintainability. We analyze the effectiveness of these models according to various aspects. To meet the goal of the research, we have identified 36 research papers. On investigating these papers, we found that various machine learning (ML), statistical (ST), and hybridized (HB) techniques have been applied to develop prediction models to predict software maintainability. The significant finding of this review is that the overall performance of ML-based models is better than that of ST models. The use of HB techniques for prediction of software maintainability is limited. The results of this review revealed that software maintainability prediction (SMP) models developed using ML techniques outperformed models developed using ST techniques. Also, the prediction performance of few models developed using HB techniques is encouraging, yet no conclusive results about the performance of HB techniques could be reported because different HB techniques are applied in a few studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Aggarwal KK, Singh Y, Chhabra JK (2002) An integrated measure of software maintainability. In: Proceedings of the annual reliability and maintainability symposium, 2002, pp 235–241

  • Aggarwal KK, Singh Y, Chandra P, Puri M, Base R (2005) Measurement of software maintainability using a fuzzy model. J Comput Sci 1(4):538–542

    Google Scholar 

  • Aggarwal KK, Singh Y, Kaur A, Malhotra R (2006) Application of artificial neural network for predicting maintainability using object- oriented metrics. Trans Eng Comput Technol 15:285–289

    Google Scholar 

  • Al Dallal J (2013) Object-oriented class maintainability prediction using internal quality attributes. Inf Softw Technol 55(11):2028–2048

    Google Scholar 

  • Al-Jamimi HA, Ahmed MA (2013) Machine learning approaches for predicting software maintainability: a fuzzy-based transparent model. IET Softw 7(6):317–326

    Google Scholar 

  • Almugrin S, Albattah W, Melton A (2016) The journal of systems and software using indirect coupling metrics to predict package maintainability and testability. J Syst Softw 121:298–310

    Google Scholar 

  • Anwar S, Ramzan M, Rauf A, Shahid AA (2010) Software maintenance prediction using weighted scenarios: an architecture perspective. International Conf Inf Sci Appl, 1–9

  • Arisholm E (2002) Dynamic coupling measures for object-oriented software. Proc Int Softw Metrics Symp 20:33–42

    Google Scholar 

  • Bakar AD, Sultan AB, Zulzalil H, Din J (2012) Applying evolution programming search based software engineering (SBSE) in selecting the best open source software maintainability metrics. In: IEEE Symposium on Computer Applications and Industrial Electronics (ISCAIE), pp 70–73

  • Bakota T, Hegedus P, Ladanyi G, Kortvelyesi P, Ferenc R, Gyimothy T (2012) A cost model based on software maintainability. In: 28th IEEE international conference on software maintenance, pp 316–325

  • Balogh G, G, Végh ÁZ, Beszédes A (2012) Prediction of software development modification effort enhanced by a genetic algorithm, SSBSE Fast Abstract track, pp 1–6

  • Bandi RK, Vaishnavi VK, Turk DE (2003) Predicting maintenance performance using object-oriented design complexity metrics. IEEE Trans Softw Eng 29(1):77–87

    Google Scholar 

  • Baqais AAB, Alshayeb M, Baig ZA (2013) Hybrid intelligent model for software maintenance prediction. Lecture Notes Eng Comput Sci 1:358–362

    Google Scholar 

  • Boehm BW (1976) Software engineering. IEEE Trans Comput 25(12):1226–1241

    MATH  Google Scholar 

  • Basgalupp MP, Barros RC, Ruiz DD, Carlos S, Brazil SP (2012) Predicting software maintenance effort through evolutionary-based decision trees. In: Proceedings of the 27th Annual ACM Symposium on Applied Computing, pp 1209–1214

  • Basili V, Briand L, Condon S (1996) Understanding and predicting the process of software maintenance releases. In: Proceedings—international conference on software engineering, pp 464–474

  • Bengtsson PO, Bosch J (1999) Architecture level prediction of software maintenance. In: Proceedings of the 3rd European conference on software maintenance and reengineering, pp 139–147

  • Bennett KH, Rajlich VT (2000) Software maintenance and evolution: a roadmap. Proc Conf Future Softw Eng 225:73–87

    Google Scholar 

  • Bhutani S, Chug A (2015) Prediction of software maintainability using neural networks. Int J Comput Sci Commun Netw 5(2):92–95

    Google Scholar 

  • Briand L, Daly W, Wust J (1999) A unified framework for coupling measurement in object-oriented systems. IEEE Trans Softw Eng 25(1):91–121

    Google Scholar 

  • Broy M, Deissenboeck F, Pizka M (2006) Demystifying maintainability. In: Proceedings of the 2006 international workshop on Software quality—WoSQ ’06, pp 21–26

  • Campbell DT, Stanley JC, Gage NL (1963) Experimental and quasi-experimental designs for research. Houghton Mifflin, Boston

    Google Scholar 

  • Chandra D (2012) Support vector approach by using radial kernel function for prediction of software maintenance effort on the basis of multivariate approach. Int J Comput Appl 51(4):21–25

    Google Scholar 

  • Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. IEEE Trans Softw Eng 20:476–493

    Google Scholar 

  • Chug A, Malhotra R (2016) Benchmarking framework for maintainability. Int J Innovat Comput Inf Control 12(2):615–634

    Google Scholar 

  • Coleman D, Ash D, Lowther B, Oman P (1994) Using metrics to evaluate software system maintainability. Computer 27(8):44–49

    Google Scholar 

  • Conte SD, Dunsmore HE, Shen VY (1986) Software engineering metrics and models. Benjamin-Cummings Publishing Co., Redwood City

    Google Scholar 

  • Dagpinar M, Jahnke JH, Canada BC (2003) Predicting Maintainability with object-oriented metrics—an empirical comparison. In: Proceedings of the 10th working conference on reverse engineering, WCRE 2003, pp 155–164

  • Daly J, Brooks A, Miller J, Roper M, Wood M (1995) The effect of inheritance on the maintainability of object-oriented software: an empirical study. Proc Int Conf Softw Maint 44:20–29

    Google Scholar 

  • De Lucia A, Pompella E, Stefanucci S (2005) Assessing effort estimation models for corrective maintenance through empirical studies. Inf Softw Technol 47(1):3–15

    Google Scholar 

  • De Lucia A, Pompella E, Stefanucci S (2002) Effort estimation for corrective software maintenance. In: Proceedings of the 14th international conference on Software engineering and Knowledge Engineering, pp 409–416

  • Deligiannis I, Shepperd M (2004) A controlled experiment investigation of an object-oriented design heuristic for maintainability. J Syst Softw 72(2):129–143

    Google Scholar 

  • Deligiannis I, Shepperd M, Roumeliotis M, Stamelos I (2003) An empirical investigation of an object-oriented design heuristic for maintainability. J Syst Softw 65(2):127–139

    Google Scholar 

  • Elish MO, Elish KO (2009) Application of TreeNet in predicting object-oriented software maintainability: a comparative study. In: Proceedings of the european conference on software maintenance and reengineering, CSMR 2009, pp 69–78

  • Elish MO, Aljamaan H (2015) Three empirical studies on predicting software maintainability using ensemble methods. Soft Comput 19(19):2511–2524

    Google Scholar 

  • Ferneley EH (1999) Design metrics as an aid to software maintenance: an empirical study. J Softw Evolut Process 11(1):55–72

    Google Scholar 

  • Fioravanti F, Nesi P (2001) Estimation and prediction metrics for adaptive maintenance effort of object-oriented systems. IEEE Trans Softw Eng 27(12):1062–1083

    Google Scholar 

  • Genero M, Genero M, Cantone G, Informatica D (2003) Building UML class diagram maintainability prediction models based on early metrics. In: Proceedings of 9th international software metrics symposium, pp 263–275

  • Haiquan Y, Gaoliang P, Wenjian L (2009) An application of case based reasoning to predict structure maintainability. Int Conf Comput Intell Softw Eng, pp 1–5

  • Halstead M (1977) Elements of software science. Elsevier, New York

    MATH  Google Scholar 

  • Harrison R, Counsell S, Nithi R (2000) Experimental assessment of the effect of inheritance on the maintainability of object-oriented systems. J Syst Softw 52(2):173–179

    Google Scholar 

  • Harrison R, Counsell S (1998) The role of inheritance in the maintainability of object-oriented systems. In: Proceedings of ESCOM, pp 449–458

  • Hatton L (2007) How accurately do engineers predict software maintenance tasks? Computer 40(2):64–69

    Google Scholar 

  • Hayes JH (2004) A metrics-based software maintenance effort model. In: Proceedings of 8th Europian conference on software maintenance and re-engineering, pp 254–258

  • Hayes JH, Zhao L (2005) Maintainability prediction: a regression analysis of measures of evolving systems. In: 21st IEEE international conference on software maintenance (ICSM’05), pp 601–604

  • Hegedus P (2013) Revealing the effect of coding practices on software maintainability. In: IEEE International Conference on Software Maintenance, pp 578–581

  • Heitlager I, Kuipers T, Visser J (2007) A practical model for measuring maintainability. In: 6th International conference on the on quality of information and communications technology, pp 30–39

  • Henderson-Sellors B (1996) Software metrics. Prentice Hall, Upper Saddle River

    Google Scholar 

  • Hirota T, Tohki M, Overstreet CM, Hashimoto M, Cherinka R (1994) An approach to predict software maintenance cost based on ripple complexity. In: Proceedings of 1st asia-pacific software engineering conference, pp 439–444

  • Hops JM, Sherif JS (1995) Development and application of composite complexity models and a relative complexity metric in a software maintenance environment. J Syst Softw 31(2):157–169

    Google Scholar 

  • IEEE: IEEE Standard Glossary of Software Engineering Terminology. IEEE Press, Piscataway (1990)

  • Jain R, Sharma A (2016) Critical insight into maintainability and reliability assessment aspects of software systems using soft computing approaches. Int J Control Theory Appl 9(20):9397–9404

    Google Scholar 

  • Jain A, Tarwani S, Chug A (2016) An empirical investigation of evolutionary algorithm for software maintainability prediction. In: 2016 IEEE Students’ conference on electrical, electronics and computer science, SCEECS-2016, pp 1–6

  • Jeet K, Dhir R, Verma H (2012) A comparative study of bayesian and fuzzy approach to assess and predict maintainability of the software using activity-based quality model. SIGSOFT Softw Eng Notes 37(3):1–9

    Google Scholar 

  • Jeet K, Dhir R (2012) Bayesian and fuzzy approach to assess and predict the maintainability of software: a comparative study. ISRN Software Engineering, pp 1–10

  • Jin C, Jin SW (2015) Prediction approach of software fault-proneness based on hybrid artificial neural network on quantum particle swarm optimization. Appl Soft Comput 35:717–725

    Google Scholar 

  • Jindal R, Malhotra R, Jain A (2015) Predicting software maintenance effort using neural networks. In: 2015 4th International Conference on Reliability, Infocom Technologies and Optimization: Trends and Future Directions, pp 1–6

  • Jin C, Liu JA (2010) Applications of support vector mathine and unsupervised learning for predicting maintainability using object-oriented metrics. In: International conference on multimedia and information technology, MMIT 2010 vol 1, pp 24–27

  • Kaur A, Kaur K (2013) Statistical comparison of modelling methods for software maintainability prediction. Int J Softw Eng Knowl Eng 23(6):743–747

    Google Scholar 

  • Kaur K, Singh H (2011) Determination of maintainability index for object oriented systems. ACM SIGSOFT Softw Eng Notes 36(2):1–6

    Google Scholar 

  • Kaur A, Kaur K, Malhotra R (2010) Soft computing approaches for prediction of software maintenance effort. Int J Comput Appl 1(16):69–75

    Google Scholar 

  • Kaur A, Kaur K, Pathak K (2014) Software maintainability prediction by data mining of software code metrics. In: 2014 International conference on data mining and intelligent computing, ICDMIC 2014, pp 1–6

  • Kewen L, Chenxi C, Wenying L, Xianghua F, Qinghua L (2014) Software defect prediction using fuzzy integral fusion based on GA-FM. Wuhan Univ J Natural Sci 19(5):405–408

    Google Scholar 

  • Khoshgoftaar TM, Szabo RM (1994a) Improving code churn predictions during the system test and maintenance phases. In: Proceedings international conference on software maintenance, ICSM 1994 vol 7, pp 58–67

  • Khoshgoftaar TM, Szabo RM (1994b) Improving neural network predictions of software quality using principal components analysis. In: Proceedings of IEEE international conference on neural networks, ICNN 1994 vol 5, pp 3295–3300

  • Kitechenham BA (2007) Guidelines for performing systematic literature review in software engineering, Technical report EBSE-2007-001, UK

  • Ku Y, Du J, Yang Y, Wang Q (2011) Estimating software maintenance effort from use cases: an industrial case study. IEEE international conference on software maintenance, ICSM, pp 482–491

  • Kumar L (2016) Maintainability prediction of web service using support vector machine with various kernel methods. Int J Syst Assur Eng Manag 8(2):205–222

    Google Scholar 

  • Kumar L, Rath SK (2016) Hybrid functional link artificial neural network approach for predicting maintainability of object-oriented software R. J Syst Softw 121:170–190

    Google Scholar 

  • Kumar L, Rath SK (2017) Software maintainability prediction using hybrid neural network and fuzzy logic approach with parallel computing concept. Int J Syst Assur Eng Manag 8(2):1487–1502

    Google Scholar 

  • Kumar V, Kumar R, Sharma A (2014) Maintainability prediction from project metrics data analysis using artificial neural network. Middle-East J Sci Res 19(10):1412–1420

    Google Scholar 

  • Kumar L, Kumar D, Kumar S (2015) Validating the effectiveness of object-oriented metrics for predicting maintainability. Procedia Comput Sci 57:798–806

    Google Scholar 

  • Kumar L, Lal S, Murthy LB (2019) Estimation of maintainability parameters for object-oriented software using hybrid neural network and class level metrics. Int J Syst Assur Eng Manag, 1–31

  • Kumar L, Rath S (2015) Neuro-Genetic Approach for predicting maintainability using Chidamber and Kemerer software metrics suite. Recent Advances in Information and Communication Technology, Springer International Publishing, pp 31–40

  • Kumar L, Rath SK (2015) Predicting object-oriented software maintainability using hybrid neural network with parallel computing concept. In: Proceedings of the 8th India software engineering conference, ISEC-2015, pp 100–109

  • Lagerström R, Johnson P (2008) Using architectural models to predict the maintainability of enterprise systems. In: 12th european conference on software maintenance and reengineering, IEEE, pp 248–252

  • Land R (2002) Measurements of Software Maintainability. In: proceedings of ARTES Graduate Student Conference

  • Li YF, Xie M, Goh TN (2008) A bayesian inference approach for probabilistic analogy based software maintenance effort estimation. In: Proceedings of the 14th IEEE pacific rim international symposium on dependable computing, PRDC 2008, pp 176–183

  • Li W, Henry S (1993) Object-oriented metrics that predict maintainability. J Syst Softw 23(2):111–122

    Google Scholar 

  • Lorenz M, Kidd J (1994) Object-oriented software metrics. Prentice-Hall, Upper Saddle River

    Google Scholar 

  • Lowther B, Oman P (1995) The application of software maintainability in industrial software systems. J Syst Softw 29(1):3–16

    Google Scholar 

  • Malhotra R (2015) A systematic review of machine learning techniques for software fault prediction. Appl Soft Comput 27:504–518

    Google Scholar 

  • Malhotra R, Chug A (2012) Software maintainability prediction using machine learning algorithms. Softw Eng nt J (SEIJ) 2(2):19–36

    Google Scholar 

  • Malhotra R, Chug A (2014a) Application of Group Method of Data Handling model for software maintainability prediction using object oriented systems. IntJ Syst Assur Eng Manag 5(2):165–173

    Google Scholar 

  • Malhotra R, Chug A (2014b) Application of evolutionary algorithms for software maintainability prediction using object-oriented metrics. In: Proceedings of the 8th international conference on bioinspired information and communications technologies, pp 348–351

  • Malhotra R, Chug A (2016) Software maintainability: systematic literature review and current trends. Int J Softw Eng Knowl Eng 26(8):1221–1253

    Google Scholar 

  • Malhotra R, Lata K (2017) An exploratory study for predicting maintenance effort using hybridized techniques. In: Proceedings of the 10th innovations in software engineering conference. ACM, pp 26–33

  • Malhotra R, Lata K (2018) On the application of cross-project validation for predicting maintainability of open source software using machine learning techniques. In: 7th IEEE International conference on reliability, infocom technologies and optimization (ICRITO), pp 175–181

  • Manchanda S, Chug A (2015) CFS based feature subset selection for software maintainance prediction. Int J Adv Found Res Comput 2(5):13–23

    Google Scholar 

  • Mie M, Thwin T, Quah T (2003) Application of neural networks for software quality prediction using object-oriented metrics. In: Proceedings of the international conference on software maintenance, ICSM 2003, vol 76, pp 116–125

  • Mishra S, Sharma A (2015) Maintainability prediction of object oriented software by using adaptive network based fuzzy system technique. Int J Comput Appl 119(9):24–27

    Google Scholar 

  • Misra SC (2005) Modelling design/coding factors that drive maintainability of software systems. Softw Q J 13(3):297–320

    Google Scholar 

  • Muthanna S, Ponnambalaml KKK, Stacey B (2000) A maintainability model for industrial software systems using design level metrics. In: Proceeding 7th working conference on reverse engineering, pp 248–256

  • Niessink F, Van Vliet H (1997)Predicting maintenance effort with function points. Int Conf Softw Maint 32–39

  • Olatunji SO (2013) Sensitivity based linear learning method and extreme learning machines compared for software maintainability prediction of object orinented software system. ICTACT J Soft Comput 3:514–523

    Google Scholar 

  • Oman P, Hagemeister J (1992) Metrics for assessing a software system’s maintainability. In: Software Maintenance procedings, pp 337–334

  • Perepletchikov M (2007) Coupling metrics for predicting maintainability in service-oriented designs. In: 18th Australian conference on software engineering, pp 329–340

  • Ping L (2010) A quantitative approach to software maintainability prediction. In: Proceedings of 2010 international forum on information technology and applications, IFITA 2010, vol 1, pp 105–108

  • Prasanth NN, Ganesh S, Dalton GA (2008) “Prediction of maintainability using software complexity analysis: an extended FRT. In: Proceedings of the 2008 international conference on computing, communication and networking, ICCCN 2008, pp 1–9

  • Prechelt L, Unger B, Philippsen M, Tichy W (1998) Re-Evaluating inheritance depth on the maintainability of object-oriented software. Int J Empir Softw Eng pp 1–16

  • Rai V, Srivastava AM, Pandey H, Singh VK (2015) Estimation of maintainability in object oriented design phase: state of the art. Int J Sci Eng Res 6(9):25–35

    Google Scholar 

  • Rana SKA, Dash Y (2012) Maintainability prediction of object-oriented software system by multilayer perceptron model. ACM SIGSOFT Softw Eng Notes 37(5):1–4

    Google Scholar 

  • Riaz M, Mendes E, Tempero E (2009) A systematic review of software maintainability prediction and metrics. In: Proceedings of the 3rd international symposium on empirical software engineering and measurement. IEEE Computer Society, pp 367–377

  • Riaz M, Mendes E, Tempero E, Sulayman M (2013) Using CBR and CART to predict maintainability of relational database-driven software applications. In: Proceedings of the 17th international conference on evaluation and assessment in software engineering—EASE ’13, p 132

  • Rizvi SWA, Khan RA (2010) Maintainability estimation model for object-oriented software in design phase. J Comput 2(4):26–32

    Google Scholar 

  • Schneberger SL (1997) Distributed computing environments: effects on software maintenance difficulty. J Syst Softw 37(2):101–116

    Google Scholar 

  • Seref B, Tanriover O (2016) Softwarecode maintainability: a litreature review. Int J Softw Eng Appl (IJSEA) 7(3):69–87

    Google Scholar 

  • Sharawat S (2012) Software maintainability prediction using neural networks. Int J Eng Res Appl (IJERA) 2(2):750–755

    Google Scholar 

  • Sharma H, Chug A (2015) Dynamic metrics are superior than static metrics in maintainability prediction: an empirical case study. In: 4th International conference on reliability, infocom technologies and optimization, ICRITO-2015, pp 2–7

  • Sheshasaayee A, Jose MR (2016) Deployment of a fuzzy XNOR based evaluative system for the maintainability features of aspect oriented systems. Int J Appl Eng Res 11(9):6637–6642

  • Singh RK, Balyan A (2012) Approach to software maintainability prediction versus performance. Int J Soft Comput Eng 2(2):51–54

  • Singh Y, Bhatia PK, Sangwan O (2009) Predicting software maintenance using fuzzy model. ACM SIGSOFT Softw Eng Notes 34(4):1–6

    Google Scholar 

  • Somerville I (2012) Software engineering. Addison-Wesley, Boston

    Google Scholar 

  • Tarwani S, Chug A (2016) Predicting maintainability of open source software using gene expression programming and bad smells. In: 5th International conference on reliability, in infocom technologies and optimization (Trends and Future Directions), ICRITO-2016, pp 452–459

  • Van Koten C, Gray AR (2006) An application of Bayesian network for predicting object-oriented software maintainability. Inf Softw Technol 48(1):59–67

    Google Scholar 

  • Wang LJ, Hu XX, Ning ZY, Ke WH (2009) Predicting object-oriented software maintainability using projection pursuit regression. In: 1st International conference on information science and engineering, ICISE 2009, pp 3827–3830

  • Wen J, Li S, Lin Z, Hu Y, Huang C (2012) Systematic literature review of machine learning based software development effort estimation models. Inf Softw Technol 54(1):41–59

    Google Scholar 

  • Ye F, Zhu X, Wang Y (2013) A new software maintainability evaluation model based on multiple classifiers combination. IEEE International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering, pp 1588–1591

  • Zavvar M, Ramezani F (2015) Measuring of software maintainability using adaptive fuzzy neural network. Int J Modern Edu Comput Sci 7(10):27–32

    Google Scholar 

  • Zhang W, Huang LG, Ng V, Ge J (2015) SMPLearner: learning to predict software maintainability. Autom Soft Eng 22(1):111–141

    Google Scholar 

  • Zhou Y, Leung H (2007) Predicting object-oriented software maintainability using multivariate adaptive regression splines. J Syst Softw 80(8):1349–1361

    Google Scholar 

  • Zhou Y, Xu B, Yuming Z, Baowen XU (2008) Predicting the maintainability of open source software using design metrics. Wuhan Univ J Natural Sci 13(1):14–20

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ruchika Malhotra.

Ethics declarations

Conflict of interest

Authors certify that this manuscript has not been submitted to more than one journal for simultaneous consideration and it has not been published previously (partly or in full). Authors certify that no funding has been received for the conduct of this study and preparation of this manuscript. This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 6.1 Quality Assessment Questionnaire

All candidate studies selected for this systematic review were evaluated on the basis of quality questions stated in Table 14. This table describes the percentage of studies in which the quality questions were addressed completely, partially or not addressed.

1.2 6.2 Research questions addressed in selected primary studies

Table 15 describes the research questions that are addressed in all selected primary studies individually.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Malhotra, R., Lata, K. A systematic literature review on empirical studies towards prediction of software maintainability. Soft Comput 24, 16655–16677 (2020). https://doi.org/10.1007/s00500-020-05005-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-020-05005-4

Keywords

Navigation