Skip to main content
SpringerLink
Log in

An empirical study of software reliability prediction using machine learning techniques

  • Original Article
  • Published:
International Journal of System Assurance Engineering and Management Aims and scope Submit manuscript

Abstract

The applications of machine learning techniques have shown remarkable improvements for the prediction of software reliability than traditional statistical techniques. In this paper, we apply some well-known machine learning methods such as artificial neural networks, support vector machines, cascade correlation neural network, decision trees and fuzzy inference system to predict the reliability of a software product. The proposed models have been evaluated using mean absolute error, root mean squared error, correlation coefficient and precision. The 16 software life cycle databases have been used for empirical studies. These databases are extracted from data and analysis center for software. A comparative analysis is performed in order to determine the importance of each method to assess the capability of software reliability prediction models. We also observe that these models may use in reliability predictions and results may be more close to the reality and precision is very effective with varied real-life failure datasets. Finally we conclude that proposed approach is more precise in its prediction capacity having better capability of generalization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Aggarwal KK, Singh Y, Kaur A, Malhotra R (2006) Investigating the effect of coupling metrics on fault proneness in object-oriented systems. Softw Qual Prof 8(4):4–16

    Google Scholar 

  • Aggarwal KK, Singh Y, Kaur A, Malhotra R (2009) Empirical analysis for investigating the effect of object-oriented metrics on fault proneness: a replicated case study. Softw Process Improv Pract 14(1):39–62

    Article  Google Scholar 

  • Chen KC, Shiang Y, Liang TZ (2008) A study of software reliability growth from the perspective of learning effects. Reliab Eng Syst Saf 93(10):1410–1421

    Article  Google Scholar 

  • Eduardo OC, Aurora TR, Silvia RV (2010) A genetic programming approach for software reliability modeling. IEEE Trans Reliab 59(1):222–230

    Article  Google Scholar 

  • Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27:861–874

    Article  Google Scholar 

  • Fei X, Ping G, Lyu MR (2005) A novel method for early software quality prediction based on support vector machine. In: Proceedings of the 16th IEEE international symposium on software reliability engineering (ISSRE’05), Beijing, China: 213–222

  • Goel AL, Okumoto K (1979) Time-dependent fault detection rate model for software and other performance measures. IEEE Trans Reliab 28(3):206–211

    Article  MATH  Google Scholar 

  • Goel B, Singh Y (2009) An empirical analysis of metrics. Softw Qual Prof 11(3):35–45

    Google Scholar 

  • Han J, Kamber M (2006) Data mining: concepts and techniques. Morgan Kaufmann, India

    MATH  Google Scholar 

  • Hanley J, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic ROC curve. Radiology 143:29–36

    Google Scholar 

  • Jung Hua L (2010) Predicting software reliability with support vector machines. In: Proceedings of 2nd international conference on computer research and development (ICCRD’10), Kuala Lumpur, Malaysia: 765-769

  • Kapur PK, Garg RB, Kumar S (1999) Contributions to hardware & software reliability. World Scientific, Singapore

    Book  MATH  Google Scholar 

  • Kapur PK, Gupta A, Jha PC, Goyal SK (2010) Software quality assurance using software reliability growth modelling: state of the art. Int J Bus Inf Syst 6(4):463–496

    Google Scholar 

  • Kapur PK, Pham Hoang, Gupta A, Jha PC (2011) Software reliability assessment with OR applications. Springer, London

    Book  MATH  Google Scholar 

  • Karunanithi N, Whitley D, Malaiya Y (1992) Prediction of software reliability using connectionist models. IEEE Trans Softw Eng 18(7):563–574

    Article  Google Scholar 

  • Kohavi R (1995) The power of decision tables. In: The eighth european conference on machine learning (ECML-95), Heraklion, Greece, pp 174–189

  • Lawson J, Wesselman S, Craig W, Scott D (2003) Simple plots improve software reliability prediction models. Qual Eng 15(3):411–417

    Article  Google Scholar 

  • Littlewood B (1979) Software reliability model for modular structure. IEEE Trans Reliab 28(3):241–246

    Article  MATH  Google Scholar 

  • Lyu M (2005) Handbook of Software reliability engineering. TMH, India.

  • Malhotra R, Singh Y, Kaur A (2009) Comparative analysis of regression and machine learning methods for predicting fault proneness models. Int J Comput Appl Technol 35(2):183–193

    Article  Google Scholar 

  • Malhotra R, Kaur A, Singh Y (2011) Empirical validation of object-oriented metrics for predicting fault proneness at different severity levels using support vector machines. Int J Syst Assur Eng Manag 1(3):269–281. doi:10.1007/s13198-011-0048-7

    Article  Google Scholar 

  • Matlab fuzzy logic toolbox: tutorials on fuzzy inference system and ANFIS using MatLab. Available at http://www.mathworks.com. Accessed 14 Feb 2011

  • Mueller JA, Lemke F (1999) Self-organizing data mining: an intelligent approach to extract knowledge from data. Dresden, Berlin

    Google Scholar 

  • Musa JD (1980) Software life cycle empirical/experience data, data & analysis center for software. Available at http://www.dacs.org. Accessed 17 Sep 2010

  • Musa JD (1998) More reliable, faster, cheaper testing with software reliability engineering. Softw Qual Prof 1(1):27–37

    Google Scholar 

  • Musa JD (2005) Software reliability engineering: making solid progress. Softw Qual Prof 7(4):5–16

    MathSciNet  Google Scholar 

  • Norman F (1997) Application of software reliability engineering for NASA space shuttle. International Symposium on Software Reliability Engineering (ISSRE) 1:71–82

    Google Scholar 

  • Ohba M (1984) Inflexion S-shaped software reliability growth models. Stochastic models in reliability theory. Springer, Berlin, pp 144–162

    Google Scholar 

  • Pham H (2006) System software reliability. Springer, London

    Google Scholar 

  • Phillip S (2003) DTReg predictive modeling software available at http://www.dtreg.com. Accessed 8 Jan 2011

  • Ping PF, Hong WC (2006) Software reliability forecasting by support vector machines with simulated annealing algorithms. J Syst Softw 79(6):747–755

    Article  Google Scholar 

  • Raj K, Ravi V (2008) Software reliability prediction using soft computing techniques. J Syst Softw 81:576–583. doi:10.1016/jss.2007.05.005

    Article  Google Scholar 

  • Ross Q (1993) C4.5: programs for machine learning. Morgan Kaufman, San Mateo

    Google Scholar 

  • Scott E, Christian L (1991) The cascade-correlation learning architecture. Tech Rep. CMU-CS-90-100, School of computer science Carnegie Mellon University, Pittsburgh

  • Singh Y, Kumar P (2010a) A software reliability growth model for three-tier client–server system. Int J Comp Appl 1(13):9–16. doi:10.5120/289-451

    MATH  Google Scholar 

  • Singh Y, Kumar P (2010b) Determination of software release instant of three-tier client server software system. Int J Softw Eng 1(3):51–62

    Google Scholar 

  • Singh Y, Kumar P (2010c) Application of feed-forward networks for software reliability prediction. ACM SIGSOFT Softw Eng Notes 35(5):1–6

    Article  Google Scholar 

  • Singh Y, Kumar P (2010d) Prediction of software reliability using feed forward neural networks. In: Proceedings of computational intelligence and software engineering (CiSE’10), Wuhan, China: 1–5. doi:10.1109/CISE.2010.5677251

  • Vapnik V (1995) Nat Stat Learn Theory. Springer, New York

    Google Scholar 

  • Weka: classification and regression software (2010). Available at http://www.cs.waikato.ac.nz. Accessed 17 Feb 2011

  • Witten IH, Frank E (2011) Data Mining: Practical machine learning tools and techniques with Java implementations, 3rd edn. Morgan Kaufman, Addison-Wesley, San Francisco

    Google Scholar 

  • Xingguo L, Yanhua S (2007) An early prediction method of software reliability based on support vector machine. In: Proceedings international conference on wireless communications, networking and mobile computing (WiCom’07), Hefei: 6075–6078

  • Yang B, Xiang L (2007) A study on software reliability prediction based on support vector machines. In: Proceedings of international conference on industrial engineering and engineering management (IEEM’07), Singapore, 1176–1180

Download references

Acknowledgments

The authors wish to thank all anonymous reviewers for their valuable suggestions and useful comments.

Author information

Authors and Affiliations

  1. CS Department, IEC-CET, Greater Noida, India

    Pradeep Kumar

  2. M.S. University of Baroda, Baroda, India

    Yogesh Singh

Authors
  1. Pradeep Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yogesh Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pradeep Kumar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kumar, P., Singh, Y. An empirical study of software reliability prediction using machine learning techniques. Int J Syst Assur Eng Manag 3, 194–208 (2012). https://doi.org/10.1007/s13198-012-0123-8

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13198-012-0123-8

Keywords

Search

Navigation