Skip to main content
SpringerLink
Log in

A replicated empirical study of a selection method for software reliability growth models

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Replications are commonly considered to be important contributions to investigate the generality of empirical studies. By replicating an original study it may be shown that the results are either valid or invalid in another context, outside the specific environment in which the original study was launched. The results of the replicated study show how much confidence we could possibly have in the original study. We present a replication of a method for selecting software reliability growth models to decide whether to stop testing and release software. We applied the selection method in an empirical study, conducted in a different development environment than the original study. The results of the replication study show that with the changed values of stability and curve fit, the selection method works well on the empirical system test data available, i.e., the method was applicable in an environment that was different from the original one. The application of the SRGMs to failures during functional testing resulted in predictions with low relative error, thus providing a useful approach in giving good estimates of the total number of failures to expect during functional testing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Ehrlich W, Lee S, Molisanim R (1990) Applying reliability measurement: a case study. IEEE Softw 7(2):56–64

    Article  Google Scholar 

  • Ehrlich W, Prasanna B, Stanpfel J, Wu J (1993) Determining the cost of a stop-test decision. IEEE Softw 10(2):33–42

    Article  Google Scholar 

  • Fenton NE, Pfleeger SL (1997) Software metrics: a rigorous and practical approach, 2nd edn. PWS Publishing Company, Boston

    Google Scholar 

  • Fujiwara T, Yamada S (2003) A testing-domain-dependent software reliability growth model for imperfect debugging environment and its evaluation of goodness-of-fit. Elec Commun Jap Part 3 86(1):11–18

    Article  Google Scholar 

  • Gaudoin O, Yang B, Xie M (2003) A simple goodness-of-fit test for the power-law process, based on the Duane plot. IEEE Trans Reliab 52(1):69–74

    Article  Google Scholar 

  • Goel AL, Okumoto L (1979) A time dependent error detection rate model for software reliability and other performance measures. IEEE Trans Reliab 28(3):206–211

    MATH  Google Scholar 

  • Huang CY (2005) Cost-reliability-optimal release policy for software reliability models incorporating improvements in testing efficiency. J Syst Softw 77(2):139–155

    Article  Google Scholar 

  • Institute of Electrical and Electronics Engineers (1990) IEEE standard glossary of software engineering terminology, IEEE Std 610.12-1990

  • International Standards Organisation (2000) Information technology—software product evaluation—quality characteristics and guidelines for their use, ISO/IEC FDIS 9126-1. Geneva, Switzerland

  • Jeske DR, Zhang X (2005) Some successful approaches to software reliability modeling in industry. J Syst Softw 74(1):85–99

    Article  Google Scholar 

  • Kececioglu D (1991) Reliability engineering handbook, vol. 2. Prentice-Hall, Englewood Cliffs, NJ

    Google Scholar 

  • Lyu MR (ed) (1996) Handbook of software reliability engineering. McGraw-Hill, New York

  • Miller J (2005) Replicating software engineering experiments: a poisoned chalice or the Holy Grail. Inf Softw Technol 47(4):233–244

    Article  Google Scholar 

  • Montgomery DC (2001) Design and analysis of experiments, 5th edn. Wiley, New York

    Google Scholar 

  • Musa J (1999) Software reliability engineering. McGraw-Hill, New York

    Google Scholar 

  • Musa J, Ackerman A (1989) Quantifying software validation: when to stop testing? IEEE Softw 6(3):19–27

    Article  Google Scholar 

  • Musa J, Iannino A, Okumoto L (1987) Software reliability measurement, prediction, application. McGraw-Hill, New York

    Google Scholar 

  • Robson C (2002) Real world research. Blackwell Publishers, UK

    Google Scholar 

  • Siegel S, Castellan NJ (1988) Nonparametric statistics for the behavioral sciences. McGraw-Hill, Singapore

    Google Scholar 

  • Stringfellow C (2000) An integrated method for improving testing effectiveness and efficiency. PhD Dissertation, Colorado State University

  • Stringfellow C, Amschler Andrews A (2002) An empirical method for selecting software reliability growth models. Empir Softw Eng 7(4):319–343

    Article  MATH  Google Scholar 

  • Wood A (1996) Predicting software reliability. IEEE Comput 29(11):69–78

    Google Scholar 

  • Wood A (1997) Software reliability growth models: assumptions vs. reality. Proceedings of the Eighth International Symposium on Software Reliability Engineering, pp136–141

  • Yamada S, Ohba M, Osaki S (1983) S-shaped reliability growth modeling for software error detection. IEEE Trans Reliab 32(5):475–478

    Article  Google Scholar 

  • Yamada S, Ohtera H, Narihisa H (1986) Software reliability growth models with testing effort. IEEE Trans Reliab 35(1):19–23

    Article  Google Scholar 

Download references

Acknowledgment

The author would like to thank Prof. Catherine Stringfellow for being generous with her time and willing to answer my questions about the selection method. Thanks also to Prof. Anneliese Amschler Andrews and Prof. Per Runeson who provided valuable comments on the paper.

Author information

Authors and Affiliations

  1. Department of Communication Systems, Lund University, Box 118, 221 00, Lund, Sweden

    Carina Andersson

Authors
  1. Carina Andersson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carina Andersson.

Additional information

Editor: Pankaj Jalote

Rights and permissions

Reprints and permissions

About this article

Cite this article

Andersson, C. A replicated empirical study of a selection method for software reliability growth models. Empir Software Eng 12, 161–182 (2007). https://doi.org/10.1007/s10664-006-9018-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-006-9018-0

Keywords

Search

Navigation