Skip to main content
SpringerLink
Log in

Joint optimization of software time-to-market and testing duration using multi-attribute utility theory

  • S.I.: Statistical Reliability Modeling and Optimization
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

An optimal software release strategy is a well-investigated issue in software reliability literature. Comprehensive testing is expected before releasing the software into the market to enhance the reliability and security of the software device. In recent years, few analysts have recommended the scheme for software projects that support releasing the software early in the market and continue the testing process for an added period in the field environment even after the software is distributed. These studies are based on one common assumption that the efficiency of the software engineers in detecting the faults occurs at a consistent rate throughout the testing phase. However, bug-identification rate may experience discontinuity at the software release time. In software engineering, the time-point at which fault detection rate changes is termed as change-point. Consequently, an alternative software release policy is proposed in the present paper, which offers a generalized framework for fault detection phenomenon using the unified approach. An extensive analysis of software time-to-market and testing duration based on cost-efficiency and reliability measures is discussed by considering the change in tester’s fault detection rate. A multi-criteria decision making technique known as multi-attribute utility theory is applied to optimize the software release policy under field-testing (FT) and no field-testing (NFT) frameworks. The relevance of the optimization problem is illustrated using a numerical example, comprising both the exponential and S-shaped bug-detection process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Arora, A., Caulkins, J. P., & Telang, R. (2006). Research note—Sell first, fix later: Impact of patching on software quality. Management Science, 52(3), 465–471.

    Google Scholar 

  • Chiu, K. C., Ho, J. W., & Huang, Y. S. (2009). Bayesian updating of optimal release time for software systems. Software Quality Journal, 17(1), 99.

    Google Scholar 

  • Dalal, S. R., & Mallows, C. L. (1988). When should one stop testing software? Journal of the American Statistical Association, 83(403), 872–879.

    Google Scholar 

  • Garmabaki, A. H., Aggarwal, A. G., Kapur, P. K., & Yadavali, V. S. S. (2012). Modeling two-dimensional software multi-upgradation and related release problem (a multi-attribute utility approach). International Journal of Reliability, Quality and Safety Engineering, 19(03), 1250012.

    Google Scholar 

  • Goel, A. L., & Okumoto, K. (1979). Time-dependent error-detection rate model for software reliability and other performance measures. IEEE Transactions on Reliability, 28(3), 206–211.

    Google Scholar 

  • Huang, C. Y. (2005a). Cost-reliability-optimal release policy for software reliability models incorporating improvements in testing efficiency. Journal of Systems and Software, 77(2), 139–155.

    Google Scholar 

  • Huang, C. Y. (2005b). Performance analysis of software reliability growth models with testing-effort and change-point. Journal of Systems and Software, 76(2), 181–194.

    Google Scholar 

  • Huang, C. Y., & Lyu, M. R. (2005). Optimal release time for software systems considering cost, testing-effort, and test efficiency. IEEE Transactions on Reliability, 54(4), 583–591.

    Google Scholar 

  • Huang, C. Y., & Lyu, M. R. (2011). Estimation and analysis of some generalized multiple change-point software reliability models. IEEE Transactions on Reliability, 60(2), 498–514.

    Google Scholar 

  • Huang, C. Y., Lyu, M. R., & Kuo, S. Y. (2003). A unified scheme of some non-homogenous poisson process models for software reliability estimation. IEEE Transactions on Software Engineering, 29(3), 261–269.

    Google Scholar 

  • Inoue, S., Ikeda, J., & Yamada, S. (2016). Bivariate change-point modeling for software reliability assessment with uncertainty of testing-environment factor. Annals of Operations Research, 244(1), 209–220.

    Google Scholar 

  • Inoue, S., Taniguchi, S., & Yamada, S. (2015). An all-stage truncated multiple change point model for software reliability assessment. International Journal of Reliability, Quality and Safety Engineering, 22(04), 1550017.

    Google Scholar 

  • Inoue, S., & Yamada, S. (2008, December). Optimal software release policy with change-point. In IEEE international conference on Industrial engineering and engineering management, 2008. IEEM 2008 (pp. 531–535). IEEE.

  • Jiang, Z., & Sarkar, S. (2003, December). Optimal software release time with patching considered. In Workshop on Information Technologies and Systems, Seattle, WA, USA.

  • Jiang, Z., Sarkar, S., & Jacob, V. S. (2012). Postrelease testing and software release policy for enterprise-level systems. Information Systems Research, 23(31), 635–657.

    Google Scholar 

  • Kapur, P. K., & Garg, R. B. (1991). Optimal release policies for software systems with testing effort. International Journal of Systems Science, 22(9), 1563–1571.

    Google Scholar 

  • Kapur, P. K., & Garg, R. B. (1992). A software reliability growth model for an error-removal phenomenon. Software Engineering Journal, 7(4), 291–294.

    Google Scholar 

  • Kapur, P. K., Goswami, D. N., Bardhan, A., & Singh, O. (2008a). Flexible software reliability growth model with testing effort dependent learning process. Applied Mathematical Modelling, 32(7), 1298–1307.

    Google Scholar 

  • Kapur, P. K., Khatri, S. K., Tickoo, A., & Shatnawi, O. (2014). Release time determination depending on number of test runs using multi attribute utility theory. International Journal of System Assurance Engineering and Management, 5(2), 186–194.

    Google Scholar 

  • Kapur, P. K., Kumar, S., & Garg, R. B. (1999). Contributions to hardware and software reliability (Vol. 3). Singapore: World Scientific.

    Google Scholar 

  • Kapur, P. K., Pham, H., Aggarwal, A. G., & Kaur, G. (2012). Two dimensional multi-release software reliability modeling and optimal release planning. IEEE Transactions on Reliability, 61(3), 758–768.

    Google Scholar 

  • Kapur, P. K., Pham, H., Anand, S., & Yadav, K. (2011a). A unified approach for developing software reliability growth models in the presence of imperfect debugging and error generation. IEEE Transactions on Reliability, 60(1), 331–340.

    Google Scholar 

  • Kapur, P. K., Pham, H., Gupta, A., & Jha, P. C. (2011b). Software reliability assessment with OR applications. London: Springer.

    Google Scholar 

  • Kapur, P. K., Shrivastava, A. K., & Singh, O. (2017). When to release and stop testing of a software. Journal of the Indian Society for Probability and Statistics, 18(1), 19–37.

    Google Scholar 

  • Kapur, P. K., Singh, V. B., Anand, S., & Yadavalli, V. S. S. (2008b). Software reliability growth model with change-point and effort control using a power function of the testing time. International Journal of Production Research, 46(3), 771–787.

    Google Scholar 

  • Kapur, P. K., Singh, V. B., Singh, O., & Singh, J. N. (2013). Software release time based on different multi-attribute utility functions. International Journal of Reliability, Quality and Safety Engineering, 20(04), 1350012.

    Google Scholar 

  • Keeney, R. L. (1971). Utility independence and preferences for multi attributed consequences. Operations Research, 19(4), 875–893.

    Google Scholar 

  • Li, X., Xie, M., & Ng, S. H. (2012). Multi-objective optimization approaches to software release time determination. Asia-Pacific Journal of Operational Research, 29(03), 1240019.

    Google Scholar 

  • Majumdar, R., Shrivastava, A. K., Kapur, P. K., & Khatri, S. K. (2017). Release and testing stop time of a software using multi-attribute utility theory. Life Cycle Reliability and Safety Engineering, 6(1), 47–55.

    Google Scholar 

  • Marquardt, D. W. (1963). An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics, 11(2), 431–441.

    Google Scholar 

  • Minamino, Y., Inoue, S., & Yamada, S. (2015). Multi-attribute utility theory for estimation of optimal release time and change-point. International Journal of Reliability, Quality and Safety Engineering, 22(04), 1550019.

    Google Scholar 

  • Minamino, Y., Inoue, S., & Yamada, S. (2016). NHPP-based change-point modeling for software reliability assessment and its application to software development management. Annals of Operations Research, 244(1), 85–101.

    Google Scholar 

  • Musa, J. D., & Okumoto, K. (1983). Software reliability models: concepts, classification, comparisons, and practice. In J. K. Skwirzynski (Ed.), Electronic systems effectiveness and life cycle costing (pp. 395–423). Berlin: Springer.

    Google Scholar 

  • Myers, G. J. (1976). Softwear reliability: Principles and practices. New York: Wiley.

    Google Scholar 

  • Ohba, M., & Yamada, S. (1984). S-shaped software reliability growth models. In International colloquium on reliability and maintainability, 4th, Tregastel, France (pp. 430–436).

  • Okumoto, K., & Goel, A. L. (1980). Optimum release time for software systems based on reliability and cost criteria. Journal of Systems and Software, 1(4), 315–318.

    Google Scholar 

  • Pham, H., Nordmann, L., & Zhang, Z. (1999). A general imperfect-software-debugging model with S-shaped fault-detection rate. IEEE Transactions on Reliability, 48(2), 169–175.

    Google Scholar 

  • Pham, H., & Zhang, X. (1999). Software release policies with gain in reliability justifying the costs. Annals of Software Engineering, 8(1–4), 147–166.

    Google Scholar 

  • SAS, S. (2004). STAT User guide, Version 9.1.2.. Cary, NC: SAS Institute Inc.

    Google Scholar 

  • Singh, O., Kapur, P. K., Shrivastava, A. K., & Kumar, V. (2015). Release time problem with multiple constraints. International Journal of System Assurance Engineering and Management, 6(1), 83–91.

    Google Scholar 

  • Subburaj, R., & Kapur, P. K. (2014). Two practical software reliability growth models for software project management. In P. K. Kapur, et al. (Eds.), Quality, reliability and Infocomm technology and industrial technological management. I.K: International Publishing House Pvt. Ltd.

    Google Scholar 

  • Tickoo, A., Verma, A. K., Khatri, S. K., & Kapur, P. K. (2016). Modeling Two-Dimensional Framework for Multi-Upgradations of a Software with Change Point. International Journal of Reliability, Quality and Safety Engineering, 23(06), 1640008.

    Google Scholar 

  • Wood, A. (1996). Predicting software reliability. Computer, 29(11), 69–77.

    Google Scholar 

  • Yamada, S. (2014). Software reliability modeling: Fundamentals and applications (Vol. 5). Tokyo: Springer.

    Google Scholar 

  • Yamada, S., Ohba, M., & Osaki, S. (1983). S-shaped reliability growth modeling for software error detection. IEEE Transactions on Reliability, 32(5), 475–484.

    Google Scholar 

  • Yamada, S., & Osaki, S. (1985). Software reliability growth modeling: Models and applications. IEEE Transactions on Software Engineering, 12, 1431–1437.

    Google Scholar 

  • Yamada, S., & Osaki, S. (1987). Optimal software release policies with simultaneous cost and reliability requirements. European Journal of Operational Research, 31(1), 46–51.

    Google Scholar 

  • Yamada, S., & Yamaguchi, M. (2016). A method of statistical process control for successful open source software projects and its application to determining the development period. International Journal of Reliability, Quality and Safety Engineering, 23(05), 1650018.

    Google Scholar 

  • Zhang, X., & Pham, H. (2002). Predicting operational software availability and its applications to telecommunication systems. International Journal of Systems Science, 33(11), 923–930.

    Google Scholar 

  • Zhao, M. (2003). Statistical reliability change-point estimation models. In H. Pham (Ed.), Handbook of Reliability Engineering (pp. 157–163). London: Springer.

    Google Scholar 

  • Zhu, M., & Pham, H. (2018). A multi-release software reliability modeling for open source software incorporating dependent fault detection process. Annals of Operations Research, 269, 773–790.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Amity Center for Interdisciplinary Research, Amity University, Noida, Uttar Pradesh, India

    P. K. Kapur

  2. Department of Operational Research, University of Delhi, Delhi, India

    Saurabh Panwar, Ompal Singh & Vivek Kumar

Authors
  1. P. K. Kapur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saurabh Panwar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ompal Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vivek Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P. K. Kapur.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kapur, P.K., Panwar, S., Singh, O. et al. Joint optimization of software time-to-market and testing duration using multi-attribute utility theory. Ann Oper Res 312, 305–332 (2022). https://doi.org/10.1007/s10479-019-03483-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-019-03483-w

Keywords

Search

Navigation