Skip to main content
SpringerLink
Log in

Multi-phase algorithm design for accurate and efficient model fitting

  • S.I.: Reliability Modeling with Applications Based on Big Data
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Recent research applies soft computing techniques to fit software reliability growth models. However, runtime performance and the distribution of the distance from an optimal solution over multiple runs must be explicitly considered to justify the practical utility of these approaches, promote comparison, and support reproducible research. This paper presents a meta-optimization framework to design stable and efficient multi-phase algorithms for fitting software reliability growth models. The approach combines initial parameter estimation techniques from statistical algorithms, the global search properties of soft computing, and the rapid convergence of numerical methods. Designs that exhibit the best balance between runtime performance and accuracy are identified. The approach is illustrated through nonhomogeneous Poisson process and covariate software reliability growth models, including a cross-validation step on data sets not used to identify designs. The results indicate the nonhomogeneous Poisson process model considered is too simple to benefit from soft computing because it incurs additional runtime with no increase in accuracy attained. However, a multi-phase design for the covariate software reliability growth model consisting of the bat algorithm followed by a numerical method achieves better performance and converges consistently, compared to a numerical method only. The proposed approach supports higher-dimensional covariate software reliability growth model fitting suitable for implementation in a tool.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Availability of data and material

Not applicable.

References

  • Afrabandpey, H., Ghaffari, M., Mirzaei, A., & Safayani, M. (2014). A novel bat algorithm based on chaos for optimization tasks. In Proc. Iranian conference on intelligent systems (pp. 1–6).

  • Alabajee, M., & Alreffaee, T. (2018). Exploring ant lion optimization algorithm to enhance the choice of an appropriate software reliability growth model. International Journal of Computer Applications, 182(4), 1–8.

    Article  Google Scholar 

  • Alneamy, J., & Dabdoob, M. (2017). The use of original and hybrid grey wolf optimizer in estimating the parameters of software reliability growth models. International Journal of Computer Applications, 167(3), 12–21.

    Article  Google Scholar 

  • Al-Saati, D., & Abd-AlKareem, M. (2013). The use of cuckoo search in estimating the parameters of software reliability growth models. International Journal of Computer Science and Information Security, 11, 22.

    Google Scholar 

  • Al-Saati, N., & Alabajee, M. (2016). On the performance of firefly algorithm in software reliability modeling. Proceedings International Journal of Recent Research and Review, 9(4), 1–9.

    Google Scholar 

  • Altaf, I., Majeed, I., & Arshid Iqbal, K. (2016). Effective and optimized software reliability prediction using harmony search algorithm. In Proc. international conference on green engineering and technologies (pp. 1–6).

  • ANSI/IEEE. (1991). Standard glossary of software engineering terminology (std-729-1991). Tech. rep., ANSI/IEEE.

  • Archetti, F., & Schoen, F. (1984). A survey on the global optimization problem: General theory and computational approaches. Annals of Operations Research, 1(2), 87–110.

    Article  Google Scholar 

  • Bansal, J., Singh, P., Saraswat, M., Verma, A., Jadon, S., & Abraham, A. (2011). Inertia weight strategies in particle swarm optimization. In Proc. world congress on nature and biologically inspired computing (pp. 633–640).

  • Bertsekas, D. (1999). Nonlinear programming. Athena, 48, 334.

    Google Scholar 

  • Bonnans, J. F., Gilbert, J. C., Lemaréchal, C., & Sagastizábal, C. A. (2006). Numerical optimization theoretical and practical aspects. Springer.

  • Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5), 1190–1208.

    Article  Google Scholar 

  • Chen, M., Wu, H., & Shyur, H. (2001). Analyzing software reliability growth model with imperfect-debugging and changepoint by genetic algorithms. In Proc. international conference on computers and industrial engineering (pp. 520–526).

  • Choudhary, A., Baghel, A. S., & Sangwan, O. P. (2018). Parameter estimation of software reliability model using firefly optimization. In Proc. data engineering and intelligent computing (pp. 407–415). Springer.

  • Choudhary, A., Baghel, A. S., & Sangwan, O. P. (2017). An efficient parameter estimation of software reliability growth models using gravitational search algorithm. International Journal of System Assurance Engineering and Management, 8(1), 79–88.

    Google Scholar 

  • Choudhary, A., Baghel, A. S., & Sangwan, O. P. (2017). Efficient parameter estimation of software reliability growth models using harmony search. IET Software, 11(6), 286–291.

    Article  Google Scholar 

  • Costa, E. O., Vergilio, S. R., Pozo, A. T. R., & Souza, G. A. (2005). Modeling software reliability growth with genetic programming. In Proceedings international symposium on software reliability engineering.

  • Costa, E. O., de Souza, G. A., Pozo, A. T. R., & Vergilio, S. R. (2007). Exploring genetic programming and boosting techniques to model software reliability. IEEE Transactions on Reliability, 56(3), 422–434.

    Article  Google Scholar 

  • Dai, Y., Xie, M., Poh, K., & Yang, B. (2003). Optimal testing resource allocation with genetic algorithm for modular software systems. Journal of Systems and Software, 66(1), 47–55.

    Article  Google Scholar 

  • Deb, K. (2015). Multi-objective evolutionary algorithms (pp. 995–1015). Springer. https://doi.org/10.1007/978-3-662-43505-2_49

  • Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2), 182–197. https://doi.org/10.1109/4235.996017

  • Dohi, T., Nishio, Y., & Osaki, S. (1999). Optimal software release scheduling based on artificial neural networks. Annals of Software Engineering, 8, 167–185.

    Article  Google Scholar 

  • Eberhart, R., & Kennedy, J. (1995). A new optimizer using particle swarm theory. In Proc. IEEE international symposium on micro machine and human science (pp. 39–43).

  • Farr, W. (1996). Software reliability modeling survey. In M. Lyu (Ed.), Handbook of software reliability engineering (vol. 222, pp. 71–117). McGraw-Hill .

  • Fletcher, R. (1986). Practical methods of optimization. Wiley.

  • Goel, A. (1982). Software reliability modelling and estimation techniques. Tech. rep., Department of Industrial Engineering and Operations Research, Syracuse University.

  • Goel, A. (1985). Software reliability models: Assumptions, limitations, and applicability. IEEE Transactions on Software Engineering, SE–11(12), 1411–1423. https://doi.org/10.1109/tse.1985.232177

  • Goldberg, D. E. (2012). Genetic algorithms in search, optimization, and machine learning. Addison-Wesley.

  • Grippo, L., Lampariello, F., & Lucidi, L. (1989). A truncated newton method with nonmonotone line search for unconstrained optimization. Journal of Optimization Theory and Applications, 60(3), 401–419.

    Article  Google Scholar 

  • Hassanien, A., & Emary, E. (2016). Swarm intelligence: Principles, advances, and applications. CRC Press.

  • Hestenes, M., & Stiefel, E. (1952). Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49(6), 409.

    Article  Google Scholar 

  • Hudaib, A., & Moshref, M. (2018). Survey in software reliability growth models: Parameter estimation and models ranking. International Journal of Computer Systems, 5(5), 11–25.

    Google Scholar 

  • Jin, C., & Jin, S. W. (2016). Parameter optimization of software reliability growth model with s-shaped testing-effort function using improved swarm intelligent optimization. Applied Soft Computing, 40, 283–291.

    Article  Google Scholar 

  • Karaboga, D., & Basturk, B. (2007). A powerful and efficient algorithm for numerical function optimization: Artificial bee colony (ABC) algorithm. Journal of Global Optimization, 39(3), 459–471.

    Article  Google Scholar 

  • Karunanithi, N., Whitley, D., & Malaiya, Y. K. (1992). Prediction of software reliability using connectionist models. IEEE Transactions on Software Engineering, 18(7), 563–574.

    Article  Google Scholar 

  • Kaswan, K., Choudhary, S., & Sharma, K. (2015). Software reliability modeling using soft computing techniques: Critical review. Journal of Information Technology and Software Engineering, 5, 144.

    Google Scholar 

  • Kumar, A., Tripathi, R.P., Saraswat, P., & Gupta, P. (2017). Parameter estimation of software reliability growth models using hybrid genetic algorithm. In Proc. IEEE international conference on image information processing (pp. 1–6).

  • Lin, J., Chou, C., Yang, C., & Tsai, H. L. (2012). A chaotic Levy flight bat algorithm for parameter estimation in nonlinear dynamic biological systems. Computer and Information Technology, 2(2), 56–63.

    Google Scholar 

  • Li, X. L., Shao, Z. J., & Qian, J. X. (2002). An optimizing method based on autonomous animats: Fish-swarm algorithm. Systems Engineering—Theory & Practice, 22(11), 32.

    Google Scholar 

  • Li, Z., Yu, M., Wang, D., & Wei, H. (2019). Using hybrid algorithm to estimate and predicate based on software reliability model. IEEE Access, 7, 84268–84283.

    Article  Google Scholar 

  • Lu, K., & Ma, Z. (2018). Parameter estimation of software reliability growth models by a modified whale optimization algorithm. In Proc. IEEE international symposium on distributed computing and applications for business engineering and science (pp. 268–271).

  • Lyu, M. (Ed.). (1996). Handbook of software reliability engineering. McGraw-Hill.

  • Lyu, M., & Nikora, A. (1992). CASRE—A computer-aided software reliability estimation tool. In Proc. of computer-aided software engineering workshop (pp. 264–275).

  • Mantegna, R. N. (1994). Fast, accurate algorithm for numerical simulation of Lévy stable stochastic processes. Physical Review E, 49(5), 4677–4683. https://doi.org/10.1103/physreve.49.4677

  • Messac, A., Ismail-Yahaya, A., & Mattson, C. (2003). The normalized normal constraint method for generating the Pareto frontier. Structural and Multidisciplinary Optimization, 25(2), 86–98.

    Article  Google Scholar 

  • Minohara, T., & Tohma, Y. (1995). Parameter estimation of hyper-geometric distribution software reliability growth model by genetic algorithms. In Proc. international symposium on software reliability engineering (pp. 324–329).

  • Mohanty, R., Ravi, V., & Patra, M. R. (2010). The application of intelligent and soft-computing techniques to software engineering problems: A review. International Journal of Information and Decision Sciences, 2(3), 233–272.

    Article  Google Scholar 

  • Mueller-Gritschneder, D., Graeb, H., & Schlichtmann, U. (2009). A successive approach to compute the bounded pareto front of practical multiobjective optimization problems. SIAM Journal on Optimization, 20(2), 915–934.

    Article  Google Scholar 

  • Nagaraju, V., Fiondella, L., Zeephongsekul, P., Jayasinghe, C., & Wandji, T. (2017). Performance optimized expectation conditional maximization algorithms for nonhomogeneous Poisson process software reliability models. IEEE Transactions on Reliability, 66(3), 722–734.

    Article  Google Scholar 

  • Nagaraju, V., Jayasinghe, C., & Fiondella, L. (2020). Optimal test activity allocation for covariate software reliability models. Journals of Systems and Software, 168, 110643.

    Article  Google Scholar 

  • Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. The Computer Journal, 7(4), 308–313.

    Article  Google Scholar 

  • Okamura, H., Watanabe, Y., & Dohi, T. (2002). Estimating mixed software reliability models based on the EM algorithm. In Proceedings international symposium on empirical software engineering (pp. 69–78).

  • Okamura, H., Watanabe, Y., & Dohi, T. (2003). An iterative scheme for maximum likelihood estimation in software reliability modeling. In Proc. international symposium on software reliability engineering (pp. 246–256).

  • Pai, P. F., & Hong, W. C. (2006). Software reliability forecasting by support vector machines with simulated annealing algorithms. Journal of Systems and Software, 79, 747–755.

    Article  Google Scholar 

  • Powell, M. J. D. (1964). An efficient method for finding the minimum of a function of several variables without calculating derivatives. The Computer Journal, 7(2), 155–162.

    Article  Google Scholar 

  • Rao, M., & Anuradha, K. (2016). A hybrid method for parameter estimation of software reliability growth model using modified genetic swarm optimization with the aid of logistic exponential testing effort function. In Proc. IEEE international conference on research advances in integrated navigation systems (pp. 1–8).

  • Shanmugam, L., & Florence, L. (2012). A comparison of parameter best estimation method for software reliability models. International Journal of Software Engineering and Applications, 3, 91–102.

    Article  Google Scholar 

  • Sharma, D.T., Pant, M., & Abraham, A. (2011). Dichotomous search in ABC and its application in parameter estimation of software reliability growth models. In Proc. world congress on nature and biologically inspired computing (pp. 207–212).

  • Sheta, A. (2006). Reliability growth modeling for software fault detection using particle swarm optimization. In Proc. IEEE congress on evolutionary computation (pp. 3071–3078).

  • Sheta, A., & Abdel-Raouf, A. (2016). Estimating the parameters of software reliability growth models using the grey wolf optimization algorithm. International Journal of Advanced Computer Science and Applications, 7(4), 499–505.

    Google Scholar 

  • Shibata, K., Rinsaka, K., & Dohi, T. (2006). Metrics-based software reliability models using non-homogeneous Poisson processes. In Proc. international symposium on software reliability engineering (pp. 52–61).

  • Shibata, K., Rinsaka, K., & Dohi, T. (2015). M-srat: Metrics-based software reliability assessment tool. International Journal of Performability Engineering, 11(4), 369–379.

    Google Scholar 

  • Tang, R., Fong, S., Yang, X., & Deb, S. (2012). Wolf search algorithm with ephemeral memory. In Proc. international conference on digital information management (pp. 165–172). https://doi.org/10.1109/ICDIM.2012.6360147

  • Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Jarrod Millman, K., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., Carey, C., Polat, İ., Feng, Y., Moore, E. W., Van der Plas, J., Laxalde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E. A., Harris, C. R., Archibald, A. M., Ribeiro, A. H., Pedregosa, F., van Mulbregt, P., & Contributors SciPy1.0, C. (2019). SciPy 1.0–fundamental algorithms for scientific computing in python.arXiv:1907.10121arXiv e-prints

  • Xie, J., Zhou, Y., & Chen, H. (2013). A novel bat algorithm based on differential operator and Lévy flights trajectory. Computational Intelligence and Neuroscience, 2013.

  • Xing, F., & Guo, P. (2005). Support vector regression for software reliability growth modeling and prediction. In Proc. advances in neural networks (pp. 925–930).

  • Yamada, S., & Osaki, S. (1983). Reliability growth models for hardware and software systems based on nonhomogeneous Poisson processes: A survey. Microelectronics Reliability, 23(1), 91–112.

    Article  Google Scholar 

  • Yang, X. S. (2009). Firefly algorithms for multimodal optimization. In International symposium on stochastic algorithms (pp. 169–178). Springer.

  • Yang, X. (2009). Suash Deb: Cuckoo search via Lévy flights. In Proc. world congress on nature biologically inspired computing (pp. 210–214). https://doi.org/10.1109/NABIC.2009.5393690

  • Yang, X. S. (2010). A new metaheuristic bat-inspired algorithm, vol. nature inspired cooperative strategies for optimization (pp. 65–74). Springer.

  • Yang, X. S. (2012). Flower pollination algorithm for global optimization. In Proc. international conference on unconventional computing and natural computation (pp. 240–249). Springer.

  • Yang, X. S. (2014). Nature-inspired metaheuristic algorithms. Elsevier.

  • Zeephongsekul, P., Jayasinghe, C., Fiondella, L., & Nagaraju, V. (2016). Maximum-likelihood estimation of parameters of NHPP software reliability models using expectation conditional maximization algorithm. IEEE Transactions on Reliability, 65(3), 1571–1583.

    Article  Google Scholar 

  • Zheng, C., Liu, X., Huang, S., & Yao, Y. (2011). A parameter estimation method for software reliability models. Procedia Engineering, 15, 3477–3481.

    Article  Google Scholar 

  • Zitzler, E., Laumanns, M., & Thiele, L. (2001). SPEA 2: Improving the strength pareto evolutionary algorithm. TIK-Report, 103.

Download references

Funding

This work was supported by National Science Foundation Award (#1749635).

Author information

Authors and Affiliations

  1. University of Massachusetts Dartmouth, 285 Old Westport Road, North Dartmouth, MA, 02747, USA

    Joshua Steakelum, Jacob Aubertine, Kenan Chen, Vidhyashree Nagaraju & Lance Fiondella

Authors
  1. Joshua Steakelum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacob Aubertine
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kenan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vidhyashree Nagaraju
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lance Fiondella
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: LF; Methodology: JS, JA, KC, VN, LF; Formal analysis and investigation: JS and LF; Writing - original draft preparation: JS, JA, KC, VN, LF; Writing - review and editing: JS and LF; Funding acquisition: LF; Resources: LF; Supervision: LF.

Corresponding author

Correspondence to Lance Fiondella.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Code availability

GitHub.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Steakelum, J., Aubertine, J., Chen, K. et al. Multi-phase algorithm design for accurate and efficient model fitting. Ann Oper Res 311, 357–379 (2022). https://doi.org/10.1007/s10479-021-04028-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-021-04028-w

Keywords

Search

Navigation