Skip to main content
Springer Nature Link
Log in

Can Software Project Maturity Be Accurately Predicted Using Internal Source Code Metrics?

  • Conference paper
  • First Online:
Machine Learning and Data Mining in Pattern Recognition (MLDM 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9729))

  • 3114 Accesses

Abstract

Predicting a level of maturity (LoM) of a software project is important for multiple reasons including planning resource allocation, evaluating the cost, and suggesting delivery dates for software applications. It is not clear how well LoM can be actually predicted – mixed results are reported that are based on studying small numbers of subject software applications and internal software metrics. Thus, a fundamental problem and question of software engineering is if LoM can be accurately predicted using internal software metrics alone?

We reformulated this problem as a supervised machine learning problem to verify if internal software metrics, collectively, are good predictors of software quality. To answer this question, we conducted a large-scale empirical study with 3,392 open-source projects using six different classifiers. Further, our contribution is that it is the first use of feature selection algorithms to determine a subset of these metrics from the exponential number of their combinations that are likely to indicate the LoM for software projects. Our results demonstrate that the accuracy of LoM prediction using the metrics is below 61% with Cohen’s and Shah’s \(\kappa<< 0.1\) leading us to suggest that comprehensive sets of internal software metrics alone cannot be used to predict LoM in general. In addition, using a backward elimination algorithm for feature location, we compute top ten most representative software metrics for predicting LoM from a total of 90 software metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res. 1, 113–141 (2001)

    MathSciNet  MATH  Google Scholar 

  2. Alves, T., Ypma, C., Visser, J.: Deriving metric thresholds from benchmark data. In: 26th ICSM 2010, pp. 1–10. IEEE, Timisoara, September 12–18, 2010

    Google Scholar 

  3. Arcuri, A., Briand, L.C.: A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: ICSE, pp. 1–10 (2011)

    Google Scholar 

  4. Bakota, T., Hegedus, P., Krtvlyesi, P., Ferenc, R., Gyimthy, T.: A probabilistic software quality model. In: 27th ICSM 2011, Williamsburg, Virginia, USA, pp. 243–252, September 25–30, 2011

    Google Scholar 

  5. Bansiya, J., Davis, C.: A hierarchical model for object-oriented design quality assessment. TSE 28(1), 4–17 (2002)

    Google Scholar 

  6. Beizer, B.: Software Testing Techniques, 2nd edn. Van Nostrand Reinhold Co., New York (1990)

    MATH  Google Scholar 

  7. Carletta, J.: Assessing agreement on classification tasks: the kappa statistic. Comput. Linguist. 22(2), 249–254 (1996)

    Google Scholar 

  8. Cataldo, M., Nambiar, S.: On the relationship between process maturity and geographic distribution: an empirical analysis of their impact on software quality. In: Proceedings of the 7th ESEC/FSE 2009, pp. 101–110. ACM, New York (2009)

    Google Scholar 

  9. Correia, J., Kanellopoulos, Y., Visser, J.: A survey-based study of the mapping of system properties to ISO/IEC 9126 maintainability characteristics. In: 25th ICSM 2009, pp. 61–70. IEEE, Edmonton, September 20–26, 2009

    Google Scholar 

  10. D’Ambros, M., Lanza, M., Robbes, R.: Evaluating defect prediction approaches: a benchmark and an extensive comparison. Empirical Softw. Engg. 17(4–5), 531–577 (2012)

    Article  Google Scholar 

  11. Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)

    Article  Google Scholar 

  12. Dubey, S.K., Rana, A., Dash, Y.: Maintainability prediction of object-oriented software system by multilayer perceptron model. SIGSOFT Softw. Eng. Notes 37(5), 1–4 (2012)

    Article  Google Scholar 

  13. Fenton, N., Krause, P., Neil, M.: Probabilistic modelling for software quality control. In: Benferhat, S., Besnard, P. (eds.) ECSQARU 2001. LNCS (LNAI), vol. 2143, p. 444. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  14. Fenton, N.E., Neil, M.: A critique of software defect prediction models. IEEE Trans. Softw. Eng. 25(5), 675–689 (1999)

    Article  Google Scholar 

  15. Fenton, N.E., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach, 2nd edn. PWS Publishing Co., Boston (1998)

    Google Scholar 

  16. Ferenc, R., Beszédes, A., Tarkiainen, M., Gyimthy, T.: Columbus - reverse engineering tool and schema for c++. In: Proceedings of the ICSM 2002, pp. 172–181. IEEE Computer Society, Washington, DC (2002)

    Google Scholar 

  17. Ferzund, J., Ahsan, S.N., Wotawa, F.: Empirical evaluation of hunk metrics as bug predictors. In: Abran, A., Braungarten, R., Dumke, R.R., Cuadrado-Gallego, J.J., Brunekreef, J. (eds.) IWSM 2009. LNCS, vol. 5891, pp. 242–254. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  18. Genero, M., Olivas, J.A., Piattini, M., Romero, F.: Using metrics to predict OO information systems maintainability. In: Dittrich, K.R., Geppert, A., Norrie, M. (eds.) CAiSE 2001. LNCS, vol. 2068, p. 388. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  19. Ghezzi, C., Jazayeri, M., Mandrioli, D.: Fundamentals of Software Engineering: Prentice Hall PTR, Upper Saddle River, NJ, USA (2002)

    Google Scholar 

  20. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  21. Hall, T., Beecham, S., Bowes, D., Gray, D., Counsell, S.: A systematic literature review on fault prediction performance in software engineering. IEEE TSE 38(6) (2012)

    Google Scholar 

  22. Humble, J., Farley, D.: Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, 1st edn. Addison-Wesley Professional (2010)

    Google Scholar 

  23. ISO/IEC. Iso/iec 9126. software engineering - product quality (2001)

    Google Scholar 

  24. Japkowicz, N., Shah, M.: Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press, New York (2011)

    Book  MATH  Google Scholar 

  25. Jones, C.: Applied Software Measurement (2nd edn.): Assuring Productivity and Quality, 3rd edn. McGraw-Hill Inc., Hightstown (2008)

    Google Scholar 

  26. Jung, H.-W., Kim, S.-G., Chung, C.-S.: Measuring software product quality: a survey of ISO/IEC 9126. IEEE Software 21(5), 88–92 (2004)

    Article  Google Scholar 

  27. Khoshgoftaar, T.M., Allen, E.B., Xu, Z.: Predicting testability of program modules using a neural network. In: Proceedings of the 3rd Symposium on ASSET 2000, pp. 57–62. IEEE Computer Society, Washington, DC (2000)

    Google Scholar 

  28. Kim, S., Zimmermann, T., Whitehead, J.E., Zeller, A.: Predicting faults from cached history. In: 29th ICSE 2007, Minneapolis, MN, USA, pp. 489–498, May 20–26, 2007

    Google Scholar 

  29. Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97(1–2), 273–324 (1997)

    Article  MATH  Google Scholar 

  30. Koller, D., Sahami, M.: Toward optimal feature selection. In: In 13th International Conference on Machine Learning, pp. 284–292 (1995)

    Google Scholar 

  31. Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE TSE 33(1), 2–13 (2007)

    Google Scholar 

  32. Mierswa, I., Scholz, M., Klinkenberg, R., Wurst, M., Euler, T.: Yale: rapid prototyping for complex data mining tasks. In. In Proceedings of the 12th ACM SIGKDD, pp. 935–940. ACM Press (2006)

    Google Scholar 

  33. Moses, J.: Learning how to improve effort estimation in small software development companies. In: 24th COMPSAC 2000, pp. 522–527. IEEE Computer Society, Washington, DC (2000)

    Google Scholar 

  34. Oman, P., Hagemeister, J.: Metrics for assessing a software system’s maintainability. In: IEEE ICSM 1992, pp. 337–344. IEEE, Orlando, November 1992

    Google Scholar 

  35. Ozkaya, I., Bass, L., Nord, R., Sangwan, R.: Making practical use of quality attribute information. IEEE Software 25(2), 25–33 (2008)

    Article  Google Scholar 

  36. Raaschou, K., Rainer, A.W.: Exposure model for prediction of number of customer reported defects. In: Proceedings of the ESEM 2008, pp. 306–308. ACM, New York (2008)

    Google Scholar 

  37. Rahman, F., Devanbu, P.: How, and why process metrics are better. In: 35th IEEE/ACM ICSE 2013, San Francisco, CA (2013)

    Google Scholar 

  38. Rahman, F., Posnett, D., Hindle, A., Barr, E., Devanbu, P.: Bugcache for inspections: hit or miss? In: 8th ESEC/FSE 2011, pp. 322–331. ACM, Szeged, September 5–9, 2011

    Google Scholar 

  39. Rosenthal, R., Rosnow, R.L.: Essentials of Behavioral Research: Methods and Data Analysis, 2nd edn. McGraw-Hill (1991)

    Google Scholar 

  40. Shah, M.: Generalized agreement statistics over fixed group of experts. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011, Part III. LNCS, vol. 6913, pp. 191–206. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  41. Stehman, S.V.: Selecting and interpreting measures of thematic classification accuracy. Remote Sensing of Environment 61(1), 77–89 (1997)

    Article  Google Scholar 

  42. Tosun, A., Bener, A., Turhan, B., Menzies, T.: Practical considerations in deploying statistical methods for defect prediction: A case study within the turkish telecommunications industry. Inf. Softw. Technol. 52(11), 1242–1257 (2010)

    Article  Google Scholar 

  43. Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer-Verlag New York Inc., New York (1995)

    Book  MATH  Google Scholar 

  44. Vogelsang, A., Fehnker, A., Huuck, R., Reif, W.: Software metrics in static program analysis. In: Dong, J.S., Zhu, H. (eds.) ICFEM 2010. LNCS, vol. 6447, pp. 485–500. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  45. Wake, S.A., Henry, S.M.: A model based on software quality factors which predictsmaintainability. Technical report, Virginia Polytechnic Institute & State University, Blacksburg, VA, USA (1988)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Illinois at Chicago, Chicago, IL, 60607, USA

    Mark Grechanik & Nitin Prabhu

  2. College of William and Mary, Williamsburg, VA, 23185, USA

    Daniel Graham & Denys Poshyvanyk

  3. Bosch Research, Palo Alto, CA, 94568, USA

    Mohak Shah

Authors
  1. Mark Grechanik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nitin Prabhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Graham
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Denys Poshyvanyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mohak Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark Grechanik .

Editor information

Editors and Affiliations

  1. IBaI, Inst of Comp Vision and applied Comp Sci, Leipzig, Sachsen, Germany

    Petra Perner

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Grechanik, M., Prabhu, N., Graham, D., Poshyvanyk, D., Shah, M. (2016). Can Software Project Maturity Be Accurately Predicted Using Internal Source Code Metrics?. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2016. Lecture Notes in Computer Science(), vol 9729. Springer, Cham. https://doi.org/10.1007/978-3-319-41920-6_59

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41920-6_59

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41919-0

  • Online ISBN: 978-3-319-41920-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics