Skip to main content
SpringerLink
Log in

Unified framework to assess software reliability and determine optimal release time in presence of fault reduction factor, error generation and fault removal efficiency

  • Original Article
  • Published:
International Journal of System Assurance Engineering and Management Aims and scope Submit manuscript

Abstract

Reliability of software products can be affected by several factors faced during the testing phase such as the generation of additional faults during the fault removal process, inefficiency or inaccuracy of team in completely removing faults, probability by which faults are removed, and difference between the number of failures and faults removed. These conditions have been termed as error generation, fault removal efficiency (FRE), imperfect debugging parameter, and fault reduction factor (FRF) respectively. In this paper, we have proposed a unified software reliability growth model (SRGM) to assess the impact of these parameters on the reliability and release schedule software. The error generation, imperfect debugging, and FRE parameters have been assumed to be constant while FRF is time-dependent modeled by exponential, Weibull, and delayed s-shaped distribution functions. These models can be represented using a single unified SRGM that reduces the difficulty of model selection. The special cases have been validated on two real-life fault datasets of Tandem computers and radar systems. Performance measures, the goodness of fit, and boxplot analysis show that the model fits the dataset very well. Further, the model that gives the best fit has been used for release planning for determining release and warranty time that minimizes development cost. The paper also analyses the sensitivity of cost parameters, the impact of change in error generation, FRE, FRF, and the reliability constraint on optimal timings. The results are highly encouraging for software managers and engineers and add value to the existing literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

The data used from Article Analysis of Discrete Software Reliability Models: IBM Federal Systems, (1980) and Software reliability growth models. Tandem technical report (1996).

References

  • Aggarwal AG, Dhaka V, Nijhawan N (2017) Reliability analysis for multi-release open-source software systems with change point and exponentiated Weibull fault reduction factor. Life Cycle Reliab Saf Eng 6(1):3–14

    Article  Google Scholar 

  • Aggarwal AG, Gandhi N, Verma V, Tandon A (2019) Multi-release software reliability growth assessment: an approach incorporating fault reduction factor and imperfect debugging. Int J Math Oper Res 15(4):446–463

    Article  MathSciNet  MATH  Google Scholar 

  • Amit I, Feitelson DG (2021) Corrective commit probability: a measure of the effort invested in bug fixing. Softw Qual J. https://doi.org/10.1007/s11219-021-09564-z

    Article  Google Scholar 

  • Bai J, Pham H (2006) Cost analysis on renewable full-service warranties for multi-component systems. Eur J Oper Res 168(2):492–508

    Article  MathSciNet  MATH  Google Scholar 

  • Blischke W (2019) Warranty cost analysis. CRC Press

    Book  Google Scholar 

  • Brooks W, Motley R (1980) Analysis of discrete software reliability models: IBM Federal Systems Div Gaithersburg MD

  • Chatterjee S, Shukla A (2017) An ideal software release policy for an improved software reliability growth model incorporating imperfect debugging with fault removal efficiency and change point. Asia-Pac J Oper Res 34(03):1740017

    Article  MathSciNet  MATH  Google Scholar 

  • Chatterjee S, Shukla A (2019) A unified approach of testing coverage-based software reliability growth modelling with fault detection probability, imperfect debugging, and change point. J Softw Evol Process 31(3):e2150

    Article  Google Scholar 

  • Chatterjee S, Saha D, Sharma A (2021) Multi-upgradation software reliability growth model with dependency of faults under change point and imperfect debugging. J Softw Evol Process 33:e2344

    Article  Google Scholar 

  • Chen Y, Singpurwalla ND (1997) Unification of software reliability models by self-exciting point processes. Adv Appl Prob. https://doi.org/10.2307/1428006

    Article  MATH  Google Scholar 

  • Das S, Sengupta D, Dewanji A (2019) Optimum release time of a software under periodic debugging schedule. Commun Stat Simul Comput 48(5):1516–1534

    Article  MathSciNet  MATH  Google Scholar 

  • Friedman MA, Tran PY, Goddard PI (1995) Reliability of software intensive systems, William Andrew

  • Frigge M, Hoaglin DC, Iglewicz B (1989) Some implementations of the boxplot. Am Stat 43(1):50–54

    Google Scholar 

  • Goel AL (1985) Software reliability models: assumptions, limitations, and applicability. IEEE Trans Softw Eng 12:1411–1423

    Article  Google Scholar 

  • Honda K, Washizaki H, Fukazawa Y (2017) Generalized software reliability model considering uncertainty and dynamics: model and applications. Int J Softw Eng Knowl Eng 27(06):967–993

    Article  Google Scholar 

  • Hsu C-J, Huang C-Y, Chang J-R (2011) Enhancing software reliability modeling and prediction through the introduction of time-variable fault reduction factor. Appl Math Model 35(1):506–521

    Article  MATH  Google Scholar 

  • Huang C-Y, Lyu MR (2005) Optimal release time for software systems considering cost, testing-effort, and test efficiency. IEEE Trans Reliab 54(4):583–591

    Article  Google Scholar 

  • Huang C-Y, Lyu MR, Kuo S-Y (2003) A unified scheme of some nonhomogenous poisson process models for software reliability estimation. IEEE Trans Softw Eng 29(3):261–269

    Article  Google Scholar 

  • Huang C-Y, Luo S-Y, Lyu MR (1999) Optimal software release policy based on cost and reliability with testing efficiency. Paper presented at the Proceedings. Twenty-Third Annual International Computer Software and Applications Conference (Cat. No. 99CB37032)

  • Iqbal J, Ahmad N, Quadri S (2013) A software reliability growth model with two types of learning. Paper presented at the 2013 International Conference on Machine Intelligence and Research Advancement

  • Jelinski Z, Moranda P (1972) Software reliability research Statistical computer performance evaluation (pp 465–484): Elsevier

  • Kapur P, Garg R (1992) A software reliability growth model for an error-removal phenomenon. Softw Eng J 7(4):291–294

    Article  Google Scholar 

  • Kapur P, Kumar J, Kumar R (2008) A unified modeling framework incorporating change-point for measuring reliability growth daring software testing. Opsearch 45(4):317–334

    Article  MathSciNet  MATH  Google Scholar 

  • Kapur P, Aggarwal AG, Kapoor K, Kaur G (2009) Optimal testing resource allocation for modular software considering cost, testing effort and reliability using genetic algorithm. Int J Reliab Qual Saf Eng 16(06):495–508

    Article  Google Scholar 

  • Kapur P, Pham H, Gupta A, Jha P (2011) Software reliability assessment with OR applications. Springer

    Book  MATH  Google Scholar 

  • Kimura M, Toyota T, Yamada S (1999) Economic analysis of software release problems with warranty cost and reliability requirement. Reliab Eng Syst Saf 66(1):49–55

    Article  Google Scholar 

  • Kumar V, Singh V, Dhamija A, Srivastav S (2018) Cost-reliability-optimal release time of software with patching considered. Int J Reliab Qual Saf Eng 25(04):1850018

    Article  Google Scholar 

  • Langberg N, Singpurwalla ND (1985) A unification of some software reliability models. SIAM J Sci Stat Comput 6(3):781–790

    Article  MathSciNet  MATH  Google Scholar 

  • Leung Y-W (1992) Optimum software release time with a given cost budget. J Syst Softw 17(3):233–242

    Article  Google Scholar 

  • Li Q, Pham H (2017) A testing-coverage software reliability model considering fault removal efficiency and error generation. PLoS ONE 12(7):e0181524

    Article  Google Scholar 

  • Liu H-W, Yang X-Z, Qu F, Shu Y-J (2005) A general NHPP software reliability growth model with fault removal efficiency

  • Malaiya YK, Von Mayrhauser A, Srimani PK (1993) An examination of fault exposure ratio. IEEE Trans Software Eng 19(11):1087–1094

    Article  Google Scholar 

  • Musa JD (1975) A theory of software reliability and its application. IEEE Trans Softw Eng 3:312–327

    Article  Google Scholar 

  • Musa JD (2004) Software reliability engineering: more reliable software, faster and cheaper. Tata McGraw-Hill Education

    Google Scholar 

  • Musa JD, Iannino A, Okumoto K (1990) Software reliability. Adv Comput 30:85–170

    Article  Google Scholar 

  • Okumoto K, Goel AL (1979) Optimum release time for software systems based on reliability and cost criteria. J Syst Softw 1:315–318

    Article  Google Scholar 

  • Pachauri B, Dhar J, Kumar A (2015) Incorporating inflection S-shaped fault reduction factor to enhance software reliability growth. Appl Math Model 39(5–6):1463–1469

    Article  MathSciNet  MATH  Google Scholar 

  • Panwar S, Kumar V, Kapur P, Singh O (2021) Software reliability prediction and release time management with coverage. Int J Qual Reliab Manag

  • Park M, Pham H (2010) Warranty cost analyses using quasi-renewal processes for multicomponent systems. IEEE Trans Syst Man Cybern Part A Syst Hum 40(6):1329–1340

    Article  Google Scholar 

  • Park M, Pham H (2012) A new warranty policy with failure times and warranty servicing times. IEEE Trans Reliab 61(3):822–831

    Article  Google Scholar 

  • Pham H (1996) A software cost model with imperfect debugging, random life cycle and penalty cost. Int J Syst Sci 27(5):455–463

    Article  MATH  Google Scholar 

  • Pham H (2000) Software reliability. Springer Science & Business Media

    MATH  Google Scholar 

  • Pham H (2005) A generalized logistic software reliability growth model. Opsearch 42(4):322–331

    Article  MATH  Google Scholar 

  • Pham H (2007) System software reliability. Springer Science & Business Media

    Google Scholar 

  • Pham H, Zhang X (1999a) A software cost model with warranty and risk costs. IEEE Trans Comput 48(1):71–75

    Article  Google Scholar 

  • Pham H, Zhang X (1999b) Software release policies with gain in reliability justifying the costs. Ann Softw Eng 8(1–4):147–166

    Article  Google Scholar 

  • Pham H, Nordmann L, Zhang Z (1999) A general imperfect-software-debugging model with S-shaped fault-detection rate. IEEE Trans Reliab 48(2):169–175

    Article  Google Scholar 

  • Priest GL (1981) A theory of the consumer product warranty. Yale Law J 90(6):1297–1352

    Article  Google Scholar 

  • Rani P, Mahapatra G (2019) A single change point hazard rate software reliability model with imperfect debugging. Paper presented at the 2019 IEEE International Systems Conference (SysCon)

  • Shrivastava AK, Kumar V, Kapur P, Singh O (2020) Software release and testing stop time decision with change point. Int J Syst Assur Eng Manag 11(2):196–207

    Article  Google Scholar 

  • Song KY, Chang IH, Pham H (2018) Optimal release time and sensitivity analysis using a new NHPP software reliability model with probability of fault removal subject to operating environments. Appl Sci 8(5):714

    Article  Google Scholar 

  • Tam C, da Costa Moura EJ, Oliveira T, Varajão J (2020) The factors influencing the success of on-going agile software development projects. Int J Project Manage 38(3):165–176

    Article  Google Scholar 

  • Verma V, Anand S, Aggarwal AG (2019). Software warranty cost optimization under imperfect debugging. Int J Qual Reliab Manag

  • Verma V, Anand S, Aggarwal AG (2020a) Reliability assessment of multi-release software system under imperfect fault removal phenomenon decision analytics applications in industry (pp 367–380): Springer

  • Verma V, Neha N, Aggarwal AG (2020b) Software release planning using grey wolf optimizer soft computing methods for system dependability (pp 1–44): IGI Global

  • Wood A (1996) Software reliability growth models. Tandem Tech Report 96(130056):900

    Google Scholar 

  • Yamada S, Osaki S (1985) Software reliability growth modeling: models and applications. IEEE Trans Softw Eng 12:1431–1437

    Article  Google Scholar 

  • Yamada S, Ohba M, Osaki S (1984) S-shaped software reliability growth models and their applications. IEEE Trans Reliab 33(4):289–292

    Article  Google Scholar 

  • Yamada S, Tokuno K, Osaki S (1992) Imperfect debugging models with fault introduction rate for software reliability assessment. Int J Syst Sci 23(12):2241–2252

    Article  MATH  Google Scholar 

  • Zhang X, Pham H (1998) A software cost model with error removal times and risk costs. Int J Syst Sci 29(4):435–442

    Article  Google Scholar 

  • Zhu M, Pham H (2018) A multi-release software reliability modeling for open source software incorporating dependent fault detection process. Ann Oper Res 269(1–2):773–790

    Article  MathSciNet  MATH  Google Scholar 

Download references

Funding

This research work has not been funded by any agency.

Author information

Authors and Affiliations

  1. Department of Operational Research, University of Delhi, Delhi, 110007, India

    Vibha Verma & Anu G. Aggarwal

  2. Shaheed Sukhdev College of Business Studies, University of Delhi, Delhi, India

    Sameer Anand

  3. Amity Center for Interdisciplinary Research, Amity University, Noida, India

    P. K. Kapur

Authors
  1. Vibha Verma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sameer Anand
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. K. Kapur
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anu G. Aggarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sameer Anand.

Ethics declarations

Conflict of interest

The authors certify that they have no conflicts of interest to declare that are relevant to the content of the article.

Human or animal rights

No humans or animals are involved in the study to declare the informed consent.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Verma, V., Anand, S., Kapur, P.K. et al. Unified framework to assess software reliability and determine optimal release time in presence of fault reduction factor, error generation and fault removal efficiency. Int J Syst Assur Eng Manag 13, 2429–2441 (2022). https://doi.org/10.1007/s13198-022-01653-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13198-022-01653-x

Keywords

Search

Navigation