A hierarchical mixture model for software reliability prediction

https://doi.org/10.1016/j.amc.2006.07.028Get rights and content

Abstract

It is important to develop general prediction models in current software reliability research. In this paper, we propose a hierarchical mixture of software reliability models (HMSRM) for software reliability prediction. This is an application of the hierarchical mixtures of experts (HME) architecture. In HMSRM, individual software reliability models are used as experts. During the training of HMSRM, an Expectation–Maximizing (EM) algorithm is employed to estimate the parameters of the model. Experiments illustrate that our approach performs quite well in the later stages of software development, and better than single classical software reliability models. We show that the method can automatically select the most appropriate lower-level model for the data and performances are well in prediction.

Introduction

Software reliability is one of a number of aspects of computer software which can be taken into consideration when determining the quality of the software. Nowadays, software reliability is widely used by such users as Alcatel, AT&T, Bellcore, CNES (France), ENEA (Italy), Ericsson Telecom, Hewlett Packard, Hitachi, IBM, NASAs Jet Propulsion Laboratory, Lockheed-Martin, Lucent Technologies, Microsoft, Mitre, Motorola, Nortel, Saab Military Aircraft, Tandem Computers, the US Air Force, and the US Marine Corps. There are over 65 published articles by users of software reliability engineering, and the number continues to grow [1].

Building good reliability models is one of the key problems in the field of software reliability. A good software reliability model should give good predictions of future failure behavior, compute useful quantities and be widely applicable. Therefore, a very important goal of current software reliability research is to develop general prediction models [2]. Existing models typically rely on assumptions about development environments, the nature of software failures and the probability of individual failure occurrences. Thus each model can be shown to perform well with a specific failure data set, but no model appears to perform well for all cases.

In recent years, many methods have been proposed to improve the quality of reliability models. Some non-parametric methods have been introduced into the field [3], [4], [5], such as Neural Network and Genetic Programming. These types of method are flexible, but they often lack theoretical basis. Other work has focused on building a unified scheme to solve the generalizability problem mentioned above [6], [7]. Such schemes are often theoretically rigorous, but some of them are not practical because there are too many parameters to estimate. Another approach aims try to use various methods to improve the estimation performance of reliability models [8]; our work belongs to this type.

In this paper, a hierarchical mixture of software reliability models (HMSRM) is proposed, which is based on the principle of “divide-and-conquer”. HMSRM can be considered as an application of the hierarchical mixtures of experts (HME) architecture [9]. The model takes the outputs of several “experts” as inputs and integrates them together effectively using gating networks. Unlike many other divide-and-conquer algorithms, HME makes use of “soft” splits of data. Thus it can employ as much input information as possible in order to make decisions.

HMSRM is designed to exploit the advantages of several classical software reliability models, which are treated as “experts”. By this means, we can get more accurate predictions. In order to adjust the “contribution” of each expert, an Expectation–Maximization (EM) algorithm [9], [12] is employed. The experimental results show that the generalization performance of HMSRM is better than individual classical software reliability models, and that it is a good approach to model selection.

Section snippets

Classical software reliability models

In the course of development of software reliability research, many models have been built to predict future failures. Some of the models are described as Nonhomogeneous Poisson Process (NHPP) models, because the mean value function M(t) represents the cumulative number of faults exposed up to time t [10]. In practice, many of the NHPP models are proved to be effective only in a particular environment. In order to construct HMSRM, we choose four classical NHPP models as the “experts” in the

HMSRM and the EM algorithm

The HMSRM we used here is based on the HME architecture. The failure data are given in pairs as X = {(x(t), d(t))}, where x(t) represents the execution time and d(t) represents the number of failures. In the training algorithm, we divide the input space into a set of sub-regions and assign each “expert” to certain regions. The bounds of the regions are adjusted dynamically by the algorithm.

The architecture of a two-level HMSRM [9], [13] is shown in Fig. 1. It has exactly the same tree-like

Experiments

The training and testing data SYS1 and CSR1 are selected from the CD-ROM of the “Handbook of Software Reliability Engineering” [11]. In data set SYS1, there are 136 samples that contain time and number of failures, while there are 397 failures and corresponding time-to-failure data in CSR1. We take the early part of the software running data to train the HMSRM.

In the experiments, we construct a two level HMSRM. At the terminal nodes sit the four single models as shown in Fig. 1. In this part,

Conclusions

In this paper, we apply the HME architecture to mix classical software-reliability models and to form a novel model, HMSRM. During the training phase, the EM algorithm is used to ensure the convergence of the training. The experimental results show that the HMSRM can select the appropriate lower-level models for the given failure data sets automatically and can produce a good prediction.

Since its prediction quality is based on the lower-level models employed HMSRM can be built with a large

Acknowledgements

This work was fully supported by a grant from the National Natural Science Foundation of China (Project No. 60275002) and a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. CUHK4205/04E).

References (13)

  • John D. Musa, More reliable software faster and cheaper, Tutorial, 16th International Symposium on Software Reliability...
  • Karunanithi Nachimuthu et al.

    Using neural networks in reliability prediction

    IEEE Transactions on Software Engineering

    (1992)
  • R. Sitte

    Comparison of software reliability growth predictions: neural networks vs parametric recalibration

    IEEE Transactions on Software Engineering

    (1999)
  • Miyoung Shin et al.

    Empirical data modeling in software engineering using radial basis functions

    IEEE Transactions on Software Engineering

    (2000)
  • Eduardo Oliveira Costa, Silvia R. Vergilio, Aurora Pozo, Gustavo Souza, Modeling software reliability growth with...
  • Chin-Yu Huang et al.

    Unified scheme of some nonhomogenous Poisson process models for software reliability estimation

    IEEE Transactions on Software Engineering

    (2003)
There are more references available in the full text version of this article.

Cited by (0)

View full text
Copyright © 2006 Elsevier Inc. All rights reserved.