Skip to main content
SpringerLink
Log in

SMPLearner: learning to predict software maintainability

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Accurate and practical software maintainability prediction enables organizations to effectively manage their maintenance resources and guide maintenance-related decision making. This paper presents SMPLearner, an automated learning-based approach to train maintainability predictors by harvesting the actual average maintenance effort computed from the code change history as well as employing a much richer set of 44 four-level hierarchical code metrics collected by static code analysis tools. We systematically evaluated SMPLearner on 150 observations partitioned from releases of eight large scale open source software systems. Our evaluation showed that SMPLearner not only outperformed the traditional 4-metric MI model but also the recent learning-based maintainability predictors constructed based on single Class-level metrics, demonstrating that single Class-level metrics were not sufficient for maintainability prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Understand is a static analysis tool for maintaining, measuring, and analyzing critical or large code bases. It calculates the four MI metrics objectively based on the code structure. It can be accessible at http://www.scitools.com/index.php.

  2. VirtualBox’s version control system is available at https://www.virtualbox.org/browser/vbox/trunk/src/VBox.

  3. A description of Logiscope is available at https://www-304.ibm.com/support/docview.wss?uid=swg24021021.

  4. See Sect. 4.2 for details on how the conversion is done and why it is needed.

  5. Spearman’s rank correlation coefficient rather than Pearson’s method is used to compute all correlation results in this paper due to its robustness and lack of assumptions about normality.

  6. http://www.statstutor.ac.uk/resources/uploaded/spearmans

References

  • Al-Kilidar, H., Cox, K., Kitchenham, B.: The use and usefulness of the ISO/IEC 9126 quality standard, In IEEE International Symposium on Empirical Software Engineering, pp. 7–17 (2005)

  • Antonellis, P., Antoniou, D., Kanellopoulos, Y., Makris, C., Theodoridis, E., Tjortjis, C., Tsirakis, N.: A data mining methodology for evaluating maintainability according to ISO/IEC-9126 software engineering Cproduct quality standard. In Special Session on System Quality and Maintainability (SQM) (2007)

  • Baggen, R., Correia, J.P., Schill, K., Visser, J.: Standardized code quality benchmarking for improvingsoftware maintainability. Software Quality Journal 20(2), 287–307 (2012)

    Article  Google Scholar 

  • Bhattacharya, P., Iliofotou, M., Neamtiu, I., Faloutsos, M.: Graph-based analysis and prediction for software evolution. ICSE, pp. 419–429 (2012)

  • Bhattacharya, P., Neamtiu, I.: Assessing programming language impact on development and maintenance: A study on C and C++. ICSE 2011, pp. 171–180 (2011)

  • Burkett, D. Klein, D.: Two Languages are Better than one (for Syntactic Parsing). In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pp. 877–886 (2008)

  • Chidamber, S.R., Kemerer, C.F.: A metrics suite for object oriented design. IEEE Transactions on Software Engineering 20(6), 476–493 (1994)

    Article  Google Scholar 

  • Coleman, D., Ash, D., Lowther, B., Oman, P.: Using Metrics to Evaluate Software System Maintainability. IEEE Computer, pp. 44–49 (1994)

  • Foss, T., Stensrud, E., Kitchenham, B., Myrtveit, I.: A simulation study of the model evaluation criterion MMRE. IEEE Trans. Softw. Eng. 29(11), 985–995 (2003)

    Article  Google Scholar 

  • Glass, R.L.: Facts and Fallacies of Software Engineering. Addison Wesley Professional, Boston (2002)

    Google Scholar 

  • Halstead, M.H.: Elements of Software Science (Operating and Programming Systems Series). Elsevier, New York (1977)

    Google Scholar 

  • Heitlager, I., Kuipers, T., Visser, J.: A practical model for measuring maintainability. In International Conference on Quality of Information and Communications Technology, pp. 30–39 (2007)

  • IEEE Std. 610.12-1990: Standard Glossary of Software Engineering Terminology. IEEE Computer Society Press, Los Alamitos (1993)

  • Kaur, A., Kaur, K.: Statistical comparison of modelling methods for software maintainability prediction. Int. J. Softw. Eng. Knowl. Eng. 23(6), 743–774 (2013)

    Article  MathSciNet  Google Scholar 

  • Kitchenham, B.A., Pickard, L.M., MacDonell, S.G., Shepperd, M.J.: What accuracy statistics really measure. IEE. Prcc Softw 148(3), 81–85 (2001)

    Article  Google Scholar 

  • Koten, C.V., Gray, A.R.: An application of bayesian network for predicting object-oriented software maintainability. Inform. Softw. Technol. 48(1), 59–67 (2006)

    Article  Google Scholar 

  • Li, W., Henry, S.: Object-oriented metrics that predict maintainability. J. Syst. Softw. 23(2), 111–122 (1993)

    Article  Google Scholar 

  • Liso, A.: Software maintainability metrics model: an improvement in the Coleman-Oman model. Crosstalk, 15–17 (2001)

  • Malhotra, R., Chug, A.: Software maintainability prediction using machine learning algorithms. Softw. Eng. Int. J. 2(2), 19–36 (2012)

    Google Scholar 

  • McCabe, T.: A complexity measure. IEEE Trans. Softw. Eng. 2(4), 308–320 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  • Mittal, H., Bhatia, P.: Software maintainability assessment based on fuzzy logic technique. ACM SIGSOFT Softw. Eng. Notes 34(3), 1–5 (2009)

    Article  Google Scholar 

  • Muthanna, S., Kontogiannis, K., Ponnambalam, K., Stacey, B.: A maintainability model for industrial software systems using design level metrics. In Proceedings of 7th Working Conference on Reverse Engineering, pp. 248–256 (2000)

  • Oman, P. W., Hagemeister, J.: Metrics for assessing a software system’s maintainability. In Proceedings of the Conference on Software Maintenance, IEEE Computer Society Press, Los Alamitos, CA, pp. 337–344 (1992)

  • Ramil, J.F., Cortazar, D.I., Mens, T.: What does it take to develop a million lines of open source code? OSS 299, 170–184 (2009)

    Google Scholar 

  • Riaz, M., Mendes, E., Tempero, E.: A systematic review of software maintainability prediction and metrics. In Proceedings of the Third International Symposium on Empirical Software Engineering and Measurement, pp. 367–377 (2009)

  • Shepperd, M., Cartwright, M., Kadoda, G.: On building prediction systems for software engineers. Empir. Softw. Eng. 5, 175–182 (2000)

    Article  MATH  Google Scholar 

  • Singh, Y., Bhatia, P.K., Sangwan, O.: Predicting software maintenance using fuzzy model. ACM SIGSOFT Softw. Eng. Notes 34(4), 1–6 (2009)

    Article  Google Scholar 

  • Welker, K.D.: The software maintainability index revisited, crosstalk. J. Def. Softw. Eng. 14, 18–21 (2001)

    Google Scholar 

  • Welker, K.D., Oman, P.W., Atkinson, G.G.: Development and application of an automated source code maintainability index. J. Softw. Maint. Evol. R 9, 127–159 (1997)

    Article  Google Scholar 

  • Yu, L.: Indirectly predicting the maintenance effort of open-source software. J. Softw. Maint. Evol. Res. Pract. 18(5), 311–332 (2006)

    Article  Google Scholar 

  • Zhou, Y., Leung, H.: Predicting object-oriented software maintainability using multivariate adaptive regression splines. J. Syst. Softw. 80, 1349–1361 (2007)

    Article  Google Scholar 

  • Zhou, Y., Xu, B.: Predicting the maintainability of open source software using design metrics. Wuhan Univ. J. Nat. Sci. 13(1), 14–21 (2008)

    Article  Google Scholar 

Download references

Acknowledgments

We thank the anonymous reviewers for their helpful comments on an earlier draft of this manuscript. This research is supported by the U.S. National Science Foundation (NSF CNS Award # 1126747). It is partially supported by the Oversea Fund of State Key Laboratory for Novel Software Technology of Nanjing University, National Natural Science Foundation, China (No. 61100039, 61021062, 61272188, 91318301), and Natural Science Foundation of Jiangsu Province, China (No. BK20131277).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Southern Methodist University, Dallas, TX, USA

    Wei Zhang & LiGuo Huang

  2. Human Language Technology Research Institute, University of Texas at Dallas, Richardson, TX, USA

    Vincent Ng

  3. State Key Laboratory for Novel Software Technology, Software Institute, Nanjing University, Nanjing, 210093, China

    Jidong Ge

Authors
  1. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. LiGuo Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vincent Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jidong Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to LiGuo Huang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Huang, L., Ng, V. et al. SMPLearner: learning to predict software maintainability. Autom Softw Eng 22, 111–141 (2015). https://doi.org/10.1007/s10515-014-0161-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10515-014-0161-3

Keywords

Search

Navigation