skip to main content
10.1145/2666581.2666588acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
Article

Software reliability analysis considering the variation of testing-effort and change-point

Published: 16 November 2014 Publication History

Abstract

It is commonly recognized that software development is highly unpredictable and software quality may not be easily enhanced after software product is finished. During the software development life cycle (SDLC), project managers have to solve many technical and management issues, such as high failure rate, cost over-run, low quality, late delivery, etc. Consequently, in order to produce robust and reliable software product(s) on time and within budget, project managers and developers have to appropriately allocate limited development- and testing-effort and time. In the past, the distribution of testing-effort or manpower can typically be described by the Weibull or Rayleigh model. Practically, it should be noticed that development environments or methods could change due to some reasons. Thus when we plan to perform software reliability modeling and prediction, these changes or variations occurring in the development process have to be taken into consideration. In this paper, we will study how to use the Parr-curve model with multiple change-points to depict the consumption of testing-effort and how to perform further software reliability analysis. The applicability and performance of our proposed model will be demonstrated and assessed through real software failure data. Experimental results are analyzed and compared with other existing models to show that our proposed model gives better predictions.

References

[1]
P. Ammann and J. Offutt. Introduction to Software Testing, Cambridge University Press, 2008.
[2]
M.R. Lyu. McGraw Hill, editor, Handbook of Software Reliability Engineering, 1996.
[3]
J. D. Musa, A. Iannino, and K. Okumoto. McGraw-Hill, editor, Software Reliability, Measurement, Prediction and Application, 1987.
[4]
N. E. Fenton and S.L. Pfleeger. Software Metrics: A Rigorous and Practical Approach, 2nd ed., PWS Publishing, 1998.
[5]
M. Xie. Software Reliability Modeling, World Scientific Publishing Company, 1991.
[6]
X. Li, Y. F. Li, and M. Xie. Reliability analysis and optimal version-updating for open source software. Information and Software Technology, 53(9):929-936, 2011.
[7]
S. Z. Ke, Measurement and Analysis of Software Reliability Model with Parr-Curve Testing-Effort Distribution and Change-Points, MS thesis, Department of Computer Science, National Tsing Hua University, Hsinchu, Taiwan, Aug. 2012.
[8]
C. T. Lin. Enhancing the accuracy of software reliability prediction through quantifying the effect of test phase transitions. Applied Mathematics and Computation, 219(5):2478-2492, 2012.
[9]
S. Li, Q. Yin, P. Guo, and M. R. Lyu. A hierarchical mixture model for software reliability prediction. Applied Mathematics and Computation, 185(2):1120-1130, 2007.
[10]
Y. S. Dai, M. Xie, Q. Long, and S.H. Ng. Uncertainty analysis in software reliability modeling by bayesian analysis with maximum-entropy principle. IEEE Trans. on Software Engineering, 33(11):781-795, 2007.
[11]
B. W. Boehm. Englewood Cliffs and Prentice-Hall, editors, Software Engineering Economics, 1981.
[12]
C. Y. Huang and S. Y. Kuo. Analysis of incorporating logistic testing-effort function into software reliability modeling. IEEE Trans. on Reliability, 51(3):261-270, 2002.
[13]
C. Y. Huang. Performance Analysis of software reliability growth models with testing-effort and change-point. Journal of Systems and Software, 76(2):181-194, 2005.
[14]
C. T. Lin and C. Y. Huang. Enhancing and measuring the predictive capabilities of testing-effort dependent software reliability models. Journal of Systems and Software, 81(6):1025-1038, 2008.
[15]
K. C. Chiu, Y. S. Huang, and T. Z. Lee. A study of software reliability growth from the perspective of learning effects. Reliability Engineering and System Safety, 93(10):1410- 1421, 2008.
[16]
P. K. Kapur, D. N. Goswami, A. Bardhan, and O. Singh. Flexible software reliability growth model with testing effort dependent learning process. Applied Mathematical Modelling, 32(7):1298-1307, 2008.
[17]
S. Yamada, J. Hishitani, and S. Osaki. Software-reliability growth with a weibull test-effort: a model and application. IEEE Trans. on Reliability, 42(1):100-106, 1993.
[18]
S. Yamada, H. Ohtera, and H. Narihisa. Software reliability growth models with testing-effort. IEEE Trans. on Reliability, 35(1):19-23, 1986.
[19]
N. Ahmad, M. G. M. Khan, and L.S. Rafi. A study of testing-effort dependent inflection S-shaped software reliability growth models with imperfect debugging. International Journal of Quality & Reliability Management, 27(1):89-110, 2010.
[20]
N. Ahmad, M. G. M. Khan, S. M. K. Quadri, and M. Kumar. Modeling and analysis of software reliability with burr type X testing-effort and release-time determination. Journal of Modelling in Management, 4(1):28-54, 2009.
[21]
S. Chatterjee, R. B. Misra, and S. S. Alam. Joint effect of test effort and learning factor on software reliability and optimal release policy. International Journal of Systems Science, 28(4):391-396, 1997.
[22]
K. Pillai and V. S. Sukumaran Nair. A model for software development effort and cost estimation. IEEE Trans. on Software Engineering, 23(8):485-497, 1997.
[23]
P. K. Kapur and R. B. Garg. Cost-Reliability Optimum release policies for a software system with testing effort. OPSEARCH, 27(2):109-116, 1990.
[24]
F. N. Parr. An alternative to the rayleigh curve model for software development effort. IEEE Trans. on Software Engineering, 6(3):291-296, 1980.
[25]
Y. P. Chang. Estimation of parameters for nonhomogeneous poisson process: software reliability with change-point model. Communications in Statistics––Simulation and Computation, 30(3):623-635, 2001.
[26]
X. Li, M. Xie, and S.H. Ng. Sensitivity analysis of release time of software reliability models incorporating testing effort with multiple change-points. Applied Mathematical Modelling, 34(11):3560-3570, 2010.
[27]
J. Zhao, H. W. Liu, G. Cui, and X. Z. Yang. Software reliability growth model with change-point and environmental function. Journal of Systems and Software, 79(11):1578-1587, 2006.
[28]
J. Yang, Z. Deng, R. Wang, and W. Hu. Web software reliability analysis with yamada exponential testing-effort. In Proceedings of the 9th IEEE International Conference on Reliability, Maintainability and Safety, ICRMS 2011, Guiyang, China, pages 760-765, 2011.
[29]
H. J. Shyur. A stochastic software reliability model with imperfect-debugging and changepoint. Journal of Systems and Software, 66(2):135-141, 2003.
[30]
M. Zhao and M. Xie. On the log-power NHPP software reliability model. In Proceedings of the 3rd IEEE International Symposium on Software Reliability Engineering, ISSRE 1992, NC, USA, pages 14-22, 1992.
[31]
M. Zhao. Change-point problems in software and hardware reliability. Communications in Statistics––Theory and Methods, 22(3):757-768, 1993.
[32]
F. Z. Zou. A change-point perspective on the software failure process. Software Testing, Verification and Reliability, 13(2):85-93, 2003.
[33]
M. Ohba. Software reliability analysis models. IBM Journal of Research and Development, 28(4):428-443, 1984.
[34]
A. Wood. Predicting software reliability. IEEE Software, 19(11):69-77, 1996.
[35]
P. N. Misra. Software reliability analysis. IBM Systems Journal, 22(3):262-279, 1983.
[36]
K. Okumoto. Customer-perceived software reliability: measurement, prediction, application. In Keynote talk at the 22nd International Symposium on Software Reliability Engineering, ISSRE 2011, available from http://2011.issre.net/sites/default/files/okumoto_data.pdf.
[37]
H. Pham. Software Reliability, Springer-Verlag, 2000.
[38]
K. Kanoun and J. C. Laprie. Software reliability trend analyzes from theoretical to practical considerations. IEEE Trans. Software Engineering, 20(9):740-747, 1994.
[39]
J. C. Laprie, K. Kanoun, C. Beounes, and M. Kaaniche. The KAT (Knowledge–Action–Transformation) approach to the modeling and evaluation of reliability and availability growth. IEEE Trans. Software Engineering, 17(4):370-382, 1991.
[40]
M. Xie, Software reliability models––Past, present and future. In Recent Advances in Reliability Theory: Methodology, Practice and Inference, pages 323-340, Birkhauser, Boston, 2000.
[41]
S. D. Conte, H. E. Dunsmore, and V. Y. Shen. Software Engineering Metrics and Models, Benjamin-Cummings Publishing Co., Inc., Redwood City, CA, 1986.
[42]
K. Pillai and V. S. Sukumaran Nair. A model for software development effort and cost estimation. IEEE Trans. on Software Engineering, 23(8):485-497, 1997.
[43]
P. L. Li, M. Shaw, J. D. Herbsleb, B. K. Ray, and P. Santhanam. Empirical Evaluation of Defect Projection Models for Widely-Deployed Production Software Systems. In Proceedings of the 12th ACM SIGSOFT International Symposium on Foundations of Software Engineering, SIGSOFT '04/FSE-12, Irvine, CA, pages 263-272, Nov 2004.
[44]
T. M. Khoshgoftaar and T. G. Woodcock. A simulation study of the performance of the akaike information criterion for the selection of software reliability growth models. In Proceedings the 27th Annual South East Region ACM Conference, pages 419-423, 1989.

Cited By

View all
  • (2021)Software Reliability Growth Models Incorporating Software Project/Application’s Characteristics as a Power Function with Change PointOptimization Models in Software Reliability10.1007/978-3-030-78919-0_2(31-51)Online publication date: 30-Sep-2021
  • (2018)Software Performance Measuring BenchmarksInternational Conference on Wireless, Intelligent, and Distributed Environment for Communication10.1007/978-3-319-75626-4_8(121-129)Online publication date: 18-Apr-2018
  • (2015)Testing-effort function for debugging in software systems and soft computing modelProceedings of the 2015 International Conference on Green Computing and Internet of Things (ICGCIoT)10.1109/ICGCIoT.2015.7380593(913-919)Online publication date: 8-Oct-2015

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
InnoSWDev 2014: Proceedings of the International Workshop on Innovative Software Development Methodologies and Practices
November 2014
114 pages
ISBN:9781450332262
DOI:10.1145/2666581
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 November 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Change points
  2. Non-homogeneous Poisson process (NHPP)
  3. Parr curve
  4. Software reliability growth model (SRGM)
  5. Testing-effort

Qualifiers

  • Article

Conference

SIGSOFT/FSE'14
Sponsor:

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2021)Software Reliability Growth Models Incorporating Software Project/Application’s Characteristics as a Power Function with Change PointOptimization Models in Software Reliability10.1007/978-3-030-78919-0_2(31-51)Online publication date: 30-Sep-2021
  • (2018)Software Performance Measuring BenchmarksInternational Conference on Wireless, Intelligent, and Distributed Environment for Communication10.1007/978-3-319-75626-4_8(121-129)Online publication date: 18-Apr-2018
  • (2015)Testing-effort function for debugging in software systems and soft computing modelProceedings of the 2015 International Conference on Green Computing and Internet of Things (ICGCIoT)10.1109/ICGCIoT.2015.7380593(913-919)Online publication date: 8-Oct-2015

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media