Modeling and analysis of reliability of multi-release open source software incorporating both fault detection and correction processes
Introduction
The software industry is growing rapidly and has become very competitive. As a result, many software developers are cutting back their schedules to ensure prompt delivery and developing new features to keep their products competitive. This is especially true for large and complex software. After a release, reported faults in previous versions will be removed and new functions may be designed to meet new requirements in new versions. Developers generally pay greater attention to balancing competition in the market, and thus risk quality because of the short software lifecycle. The upgradation process constitutes a challenge for software companies looking to produce highly reliable software and ensure the release time is on schedule.
Over the past four decades, researchers have studied a variety of methods to assess software reliability. One of the most widely investigated and applied of those methods is the software reliability growth model (SRGM) (Lyu, 2007, Amin et al., 2013, Febrero et al., 2014, Yamada, 2014). Most SRGMs utilize the fault data collected during the test process to describe the stochastic behavior of the software fault detection process (FDP) with respect to time, and it is reasonable to assume that the fault counts in each time interval are mutually independent of each other (Amin et al., 2013). Non-homogeneous Poisson process (NHPP) model is considered as one of the most effective models (Goel and Okumoto, 1979, Lyu, 1996, Ohishi et al., 2009). They have been successfully applied in many software projects to manage tests and predict operational reliability (Jeske and Zhang, 2007, Lin and Huang, 2008, Rana et al., 2014). They have also been utilized in making critical decisions, such as those involved in cost-benefit analysis, resource allocation, and release-time determination (Peng et al., 2013, Park and Baik, 2015, Wang et al., 2015).
Furthermore, a number of specific SRGMs have been proposed for investigating the reliability of Open Source software (OSS), which is a growing area of software development and applications. For example, Tamura and Yamada (2013) propose a method of software reliability assessment for the embedded OSS with flexible hazard rate modeling. Pachauri et al. (2013) blended fuzzy set theory with software reliability measurement and total cost analysis, and Gratus and Pratibha (2013) proposed an approach for carrying out pre-statistical data analyses based on assessment of software's reliability metrics. Luan and Huang (2014) proposed an improved Pareto distribution model for analyzing the failure process, although their method is confined to ungrouped data.
In this paper, the failure process in testing multi-release software is further explored by taking into consideration a delay in the fault repair time based on the time-delay model proposed by Wu et al. (2007). Both fault-correction and detection processes are considered. It is assumed that the faults in a new version comprise both undetected faults in a previous version and new faults introduced during the development process of the new version. A framework for assessing the expected number of remaining faults in each version is proposed and the optimal release time for each version is also investigated.
The remainder of this paper is organized as follows. Section 2 outlines the proposed framework for multi-release software modeling. In Section 3, the parameter estimation with Least Square Estimation (LSE) is developed and the optimal release strategy for such software is discussed. Section 5 demonstrates the application of the proposed models with a three-release dataset collected from a practical OSS test process and presents the results of optimal release time analysis. Finally, the conclusion is given in Section 6.
Section snippets
Modeling the multi-release situation
Most of the existing SRGMs focus on the software development process of only a single version. It is thus necessary to investigate changes in reliability arising from ongoing releases, which is a rather complex problem as usually there are many reasons for a new release. Several studies have been carried out in this regard in the literature. For example, Smidts et al. (1998) applied software failure data from a previous release to perform reliability estimation on a current release, and
Single-release modeling framework for FDP and FCP
For single-version software, the method of modeling FDP is like the traditional NHPP SRGM in which the cumulative number of detected faults, N(t), is assumed to follow a Poisson distribution with mean value function (MVF) md(t), i.e.,
According to the basic assumption of fault removal, the MVF can be given by where λd(t) refers to the failure rate during the test process and F(t) is a cumulative distribution function. In
Parameter estimation
Nonlinear LSE is applied to estimate the model parameters, and those estimates are the optimal solution to the following nonlinear programming problem: where nij and mij refer to the observed cumulative fault counts in the FDP and FCP of version j by testing time ti, and θ constitutes the model parameters. The foregoing extreme problem can be solved with the build-in function in MATLAB.
Optimal upgrading planning problem for multi-release software
When a company is planning to develop
Numerical illustration
In this section, two numerical application examples are given for the illustration purpose. The applied datasets are from online bug tracking systems (Mozilla and Gnome). Many organizations, particularly enterprises using OSS, utilize such systems in managing their software development processes.
In the numerical example, only those faults that are not duplicate and are reproducible for others are selected. The detection time of each fault is the opening time of this fault record, and the
Discussion
In this paper, a framework for modeling multi-release software reliability is first developed and parameter estimation problem in this situation is also studied. The developed approach takes into consideration delays in fault repair time based on the time delay model. The research incorporates software upgrading that has been an efficient approach to dealing with the increasing competition in today's market. It is assumed that the faults in a new software version comprise both undetected faults
Acknowledgments
The authors sincerely thank the editor and the reviewers of an earlier version of this paper. The comments have led to significant improvement in the presentation and additional research carried out.
Jianfeng Yang is an Associate Professor in Guizhou University. He received his Ph.D. degree in software engineering and reliability analysis in Guizhou University. His research interests include reliability engineering and web system reliability and availability, and applied statistics.
References (36)
- et al.
An approach to software reliability prediction based on time series modeling
J. Syst. Softw.
(2013) - et al.
A systematic mapping study of software reliability modeling
Inf. Softw. Technol.
(2014) - et al.
Reliability growth by failure mode removal
Reliab. Eng. Syst. Saf.
(2014) - et al.
Robust recurrent neural network modeling for software fault detection and correction prediction
Reliab. Eng. Syst. Saf.
(2007) - et al.
Software reliability analysis and assessment using queueing models with multiple change-points
Comput. Math. Appl.
(2010) - et al.
Assessing the validity of one-part software reliability models using likelihood ratio and early detection tests
J. Syst. Softw.
(2007) - et al.
Reliability analysis and optimal version-updating for open source software
Inf. Softw. Technol.
(2011) - et al.
Enhancing and measuring the predictive capabilities of testing-effort dependent software reliability models
J. Syst. Softw.
(2008) - et al.
An integration of fault detection and correction processes in software reliability analysis
J. Syst. Softw.
(2006) - et al.
Gompertz software reliability model: estimation algorithm and empirical validation
J. Syst. Softw.
(2009)
Incorporating inflection S-shaped fault reduction factor to enhance software reliability growth
Appl. Math. Model.
Modeling optimal release policy under fuzzy paradigm in imperfect debugging environment
Inf. Softw. Technol.
Improving software reliability prediction through multi-criteria based dynamic model selection and combination
J. Syst. Softw.
Testing effort dependent software reliability model for imperfect debugging process considering both detection and correction
Reliab. Eng. Syst. Saf.
Selecting software reliability growth models and improving their predictive accuracy using historical projects data
J. Syst. Softw.
An imperfect software debugging model considering log-logistic distribution fault content function
J. Syst. Softw.
Uncertainty analysis in software reliability modeling by Bayesian approach with maximum-entropy principle
IEEE Trans. Softw Eng.
Multi up-gradation software reliability growth model with faults of different severity
Cited by (77)
SDE-based software reliability additive models with masked data using ELS algorithm
2024, Journal of King Saud University - Computer and Information SciencesNASH EQUILIBRIUM STRATEGIES REVISITED IN SOFTWARE RELEASE GAMES
2022, Probability in the Engineering and Informational SciencesSOFTWARE RELIABILITY BASED ON RENEWAL PROCESS MODELING FOR ERROR OCCURRENCE DUE TO EACH BUG WITH PERIODIC DEBUGGING SCHEDULE
2022, Probability in the Engineering and Informational SciencesA Study on Optimal Release Schedule for Multiversion Software
2024, INFORMS Journal on ComputingA model of software fault detection and correction processes considering heterogeneous faults
2023, Quality and Reliability Engineering InternationalSoftware reliability model for open-source software that considers the number of finite faults and dependent faults
2023, Mathematical Biosciences and Engineering
Jianfeng Yang is an Associate Professor in Guizhou University. He received his Ph.D. degree in software engineering and reliability analysis in Guizhou University. His research interests include reliability engineering and web system reliability and availability, and applied statistics.
Yu Liu is a Ph.D. student in the department of Systems Engineering and Engineering Management, City University of Hong Kong. His research interests include software reliability growth model, reliability prediction for software/hardware system. Liu received a MS in nuclear engineering from Tsinghua University.
Min Xie is an elected fellow of IEEE and a Chair Professor of Industrial Engineering at City University of Hong Kong. He received his PhD in Quality Technology in 1987 from Linkoping University in Sweden. He was awarded the prestigious LKY research fellowship in 1991 and he is a Professor at National University of Singapore.
Ming Zhao is a Ph.D. supervisor at Guizhou University. He received his B.S. degree in mathematics from Guizhou University in 1982, M.S. degree in statistics from Wuhan University, China, in 1986, and Ph.D. degree in industrial engineering from Linkoping University, Sweden, 1994. Before he became a Ph.D. candidate at Linkoping University, he had served three years as a reliability engineer at China Academy of Launch Vehicle Technology, Beijing, China. In 1997, he joined the Department of Technology, University of Gavle, Sweden. Since 2002, his research interests include reliability engineering, software quality engineering, and applied statistics.