SREPT: software reliability estimation and prediction tool☆
Introduction
Software is an integral part of many critical and non-critical applications, and virtually any industry is dependent on computers for their basic functioning. As computer software permeates our modern society, and will continue to do so at an increasing pace in the future, the assurance of its quality becomes an issue of critical concern. Techniques to measure and ensure reliability of hardware have seen rapid advances, leaving software as the bottleneck in achieving overall system reliability.
Software reliability is defined as the probability of failure-free software operation for a specified period of time in a specified environment [43]. Its evaluation includes two types of activities [32]: reliability estimation and reliability prediction.
- •
Estimation. This determines the current software reliability [9] by applying statistical inference techniques to failure data obtained during system test or during system operation. This is a measure of “achieved” reliability, and determines whether a reliability growth model is a good fit in retrospect.
- •
Prediction. This determines future software reliability based upon a fitted reliability growth model or other available software metrics and measures.
Various approaches have been proposed to achieve reliable software systems and these may be classified as [26]
- •
Fault prevention. To avoid, by construction, fault occurrences.
- •
Fault removal. To detect, by verification and validation, the existence of faults and eliminate them.
- •
Fault tolerance. To provide, by redundancy, service complying with the specification in spite of faults having occurred or occurring.
- •
Fault/failure forecasting. To estimate, by evaluation, the presence of faults and the occurrence and consequences of failures.
Of these, fault/failure forecasting techniques have received widespread attention with the development of software reliability growth models [6]. These models capture the process of fault detection and removal during the testing phase in order to forecast failure occurrences and hence software reliability during the operational phase. Many of these models have been encapsulated into tools [32] like SMERFS, AT&T SRE Toolkit, SoRel, CASRE [31], RAT [28], M-élopée [45]. The problem with applying such tools effectively towards improving the quality of software is that they need data about the failures detected during the testing phase in order to estimate software reliability model parameters. Thus they assess reliability very late in the life-cycle of the software, and it may be too costly to take remedial actions if the software does not meet the desired reliability objective. Also, these tools do not take into account the architecture of the software. Though some techniques have been proposed [22], [25], [36], [40] to perform architecture-based software reliability prediction, there is no special purpose tool available yet. Thus, there is a need for a tool that can track the quality of a software product and provide insights throughout the life-cycle of the software. In this paper we present the high level design of a tool which provides a unified framework encompassing several techniques to monitor the reliability of a software product during the entire software development process.
The rest of the paper is organized as follows: in Section 2 we provide the motivation for a new tool. Section 3 presents the high-level design of SREPT. Section 4 provides an illustration of the use of SREPT in estimating software reliability. In Section 5 we comment on SREPT’s expected impact in the area of software reliability estimation and prediction. Section 6 concludes the paper by summarizing the features of SREPT that are presented in this paper.
Section snippets
Motivation
For state of the art research efforts to become best current practices in the industry, they ought to be made available in a systematic, user-friendly form. This factor has motivated the development of several tools for software reliability estimation and prediction. These tools can be broadly categorized into the following groups:
- •
Tools which use software metrics (product/process metrics) at the end of the development phase as inputs and either classify the modules into fault-prone or
Design and architecture of SREPT
This section presents the high-level design of SREPT. The block diagram in Fig. 1 depicts the architecture of SREPT. As can be seen from the figure, SREPT supports the following two approaches to software reliability prediction — the black-box-based and the architecture-based approaches.
Illustration
This section illustrates the use of SREPT in software reliability estimation and prediction.
Expected impact of SREPT
Fig. 18 shows SREPT’s position in the life-cycle of a software product. As can be seen from the figure, SREPT offers several techniques that can be used at various stages in the software life-cycle, according to a waterfall model [1]. Thus it makes it possible to monitor the quality of the entire software development process under a unified framework for software reliability estimation and prediction. We expect that this aspect along with its ability to perform architecture-based predictions
Conclusion
In this paper we presented the high-level design of a tool offering a unified framework for software reliability estimation that can be used to assist in evaluating the quality of a software product through its development process, right from the architectural phase all the way up to the operational phase. This is because the tool implements several software reliability techniques including software product/process metrics-based techniques used in the pre-test phase, interfailure times-based
Acknowledgements
This paper is an extended version of a paper [38] that appeared in the Proceedings of the TOOLS’98 Conference, published in Lecture Notes in Computer Science, vol. 1469 by Springer.
Srinivasan Ramani received the B.E. degree in Computer Science and Engineering from Sri Venkateswara College of Engineering, Sriperumbudur, India, in May 1995, and the M.S. degree in Electrical and Computer Engineering from Duke University, Durham, USA in 1998. He is currently working towards a Ph.D. degree in Electrical and Computer Engineering at Duke University. His research interests include software reliability, software performance and distributed computer systems.
References (49)
A spiral model of software development and enhancement
IEEE Comput.
(1988)- S. Brocklehurst, B. Littlewood, Techniques for prediction analysis and recalibration, in: M.R. Lyu (Ed.), Handbook of...
- et al.
Some graphical aids for deciding when to stop testing software
IEEE J. Selected Areas Commun.
(1990) - et al.
When to stop testing for large software systems with changing code
IEEE Trans. Software Eng.
(1994) - J.A. Denton, ROBUST, an integrated software reliability tool, Technical Report, Colorado State University,...
- W. Farr, Software reliability modeling survey, in: M.R. Lyu (Ed.), Handbook of Software Reliability Engineering,...
- W.H. Farr, O. Smith, Statistical Modeling and Estimation of Reliability Functions for Software (SMERFS) User’s Guide,...
- et al.
Time-dependent error-detection rate models for software reliability and other performance measures
IEEE Trans. Reliability
(1979) - S. Gokhale, P.N. Marinos, K.S. Trivedi, Important milestones in software reliability modeling, in: Proceedings of the...
- S. Gokhale, T. Philip, P.N. Marinos, K.S. Trivedi, Unification of finite failure NHPP models through test coverage, in:...
Functional program testing
IEEE Trans. Software Eng.
Cited by (24)
A covariate software tool to guide test activity allocation
2022, SoftwareXW-SRAT: Wavelet-based Software Reliability Assessment Tool
2021, IEEE International Conference on Software Quality, Reliability and Security, QRSComparative Study of Open Source Software Reliability Assessment Tools
2020, Proceedings of 2020 IEEE International Conference on Artificial Intelligence and Information Systems, ICAIIS 2020Functionality parameters of different software reliability tools
2020, International Journal of Advanced Science and Technology
Srinivasan Ramani received the B.E. degree in Computer Science and Engineering from Sri Venkateswara College of Engineering, Sriperumbudur, India, in May 1995, and the M.S. degree in Electrical and Computer Engineering from Duke University, Durham, USA in 1998. He is currently working towards a Ph.D. degree in Electrical and Computer Engineering at Duke University. His research interests include software reliability, software performance and distributed computer systems.
Swapna S. Gokhale received a B.E.(Hons.) in Electrical Engineering and Computer Science from Birla Institute of Technology and Science, Pilani, India, in June 1994, and M.S. and Ph.D. in Electrical and Computer Engineering from Duke University in 1996 and 1998, respectively. Currently, she is a PostGraduate Researcher at the University of California, Riverside. Her research interests include software testing and reliability, software performance, program comprehension and software maintenance, and multimedia and wireless networking.
Kishor S. Trivedi received the B.Tech. degree from the Indian Institute of Technology (Bombay), and M.S. and Ph.D. degrees in Computer Science from the University of Illinois, Urbana-Champaign. He is the author of a well known text entitled, “Probability and Statistics with Reliability, Queuing and Computer Science Applications”, published by Prentice-Hall. He has recently published two books entitled, “Performance and Reliability Analysis of Computer Systems”, published by Kluwer Academic Publishers and “Queueing Networks and Markov Chains” by Wiley. His research interests are in reliability and performance assessment of computer and communication systems. He has published over 250 papers and lectured extensively on these topics. He has supervised 32 Ph.D. dissertations. He is a Fellow of the Institute of Electrical and Electronics Engineers. He is a Golden Core Member of IEEE Computer Society. He is a Professor in the Department of Electrical and Computer Engineering at Duke University, Durham, NC. He also holds a joint appointment in the Department of Computer Science at Duke. He is the Duke-Site Director of an NSF Industry–University Cooperative Research Center between NC State University and Duke University for carrying out applied research in computing and communications. He has been on the Duke faculty since 1975. He was an Editor of the IEEE Transactions on Computers from 1983 to 1987. He is a co-designer of HARP, SAVE, SHARPE and SPNP modeling packages. These packages have been widely circulated.
- ☆
Supported in part by a contract from Charles Stark Draper Laboratory and in part by Bellcore as a core project in the Center for Advanced Computing and Communication.