Abstract
In typical software development, a software reliability growth model (SRGM) is applied in each testing activity to determine the time to finish the testing. However, there are some cases in which the SRGM does not work correctly. That is, the SRGM sometimes mistakes quality for poor quality products. In order to tackle this problem, we focussed on the trend of time series data of software defects among successive testing phases and tried to estimate software quality using the trend. First, we investigate the characteristics of the time series data on the detected faults by observing the change of the number of detected faults. Using the rank correlation coefficient, the data are classified into four kinds of trends. Next, with the intention of estimating software quality, we investigate the relationship between the trends of the time series data and software quality. Here, software quality is defined by the number of faults detected during six months after shipment. Finally, we find a relationship between the trends and metrics data collected in the software design phase. Using logistic regression, we statistically show that two review metrics in the design and coding phase can determine the trend.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Basin, S.L. 1973. Estimation of software error rates via captrue-recapture sampling, Technical report, Science Application, Inc.
Bisant, D.B. and Lyle, J.R. 1989. A two-person inspection method to improve programming productivity, IEEE Trans. on Software Engineering {15}(10): 1294–1304.
Broekman, B. and Notenboom, E. 2002. Testing Embedded Software, Addison-wesley.
Compton, T. and Withrow, C. 1990. Prediction and control of Ada software defects, Journal of Systems and Software {12}: 199–207.
Fagan, M.E. 1986. Advances in Software inspections, IEEE Trans. on Software Engineering {12}(7): 744–751.
Goel, A.L. 1985. Software reliability models: Assumptions, limitations, and applicability, IEEE Trans. on Software Engineering, 1411–1423.
Halstead, M.H. 1977. Elements of Software Science, Elsevier.
Hochberg, Y. and Tamhane, A.C. 1987, Multiple Comparison Procedures, Wiley.
Horch, J.W. 2003, Practical Guide to Software Quality Management, Artech House Publishers.
International Standard Organization. 1990. IEEE Standard Glossary of Software Engineering Terminology. IEEE Std 610.12–1990.
Kendall, M. and Gibbons, J.D. 1990. Rank Correlation Methods, 5th ed., Edward Arnold.
Khoshgoftaar, T.M. and Seliya, N. 2002. Tree-based software quality estimation models for fault prediction, In Proc. 8th IEEE International Symposium on Software Metrics, pp. 203–214.
Lehmann, E.L. 1975. Nonparametrics: Statistical Methods Based on Ranks, Holden–Day, Inc.
Lyu, M.R. (ed.). 1996. Handbook of Software Reliability Engineering, McGraw Hill.
Marick, B. 1995. The Craft of Software Testing: Subsystem Testing Including Object-Based and Object-Oriented Testing, Prentice-Hall, NJ.
Marks, D.M. 1992. Testing Very Big Systems, McGraw-Hill.
Mizuno, O., Shigematsu, E., Takagi, Y., and Kikuno, T. 2002. On Estimating testing effort needed to assure field quality in software development, In Proc. of 13th International Symposium on Software Relaibility Engineering (ISSRE2002), Annapolis, MD, USA, pp. 139–146.
Morgan, J.A. and Knafl, G.J. 1996. Residual fault density prediction using regression methods, In Proc. 7th International Symposium on Software Reliability Engineering, pp. 87–92.
Munson, J.C. and Khoshgoftaar, T.M. 1992. The detection of fault–prone programs, IEEE Trans. on Software Engineering {18}(5): 423–433.
Musa, J.D., Iannino, A., and Okumoto, K. 1987. Software Reliability: Measurement, Prediction, Application, McGraw-Hill.
Muto, S. 1995. Statistical Analysis Handbook, 1st ed., Asakura Books (in Japanese).
Padberg, F., Ragg, T., and Schoknecht, R. 2004. Using machine learning for estimating the defect content after an inspection,IEEE Trans. on Software Engineering {30}(1): 17–28.
Paulk, M.C., Curtis, B., and Weber, C. 1993. Capability maturity model, version 1.1, IEEE Software {10}(4): 18–27.
Schnieidewind, N.F. 1997. Software metrics model for integrating quality control and prediction, In Proc. 8th International Symposium on Software Reliability Engineering, pp. 402–415.
Smidts, C., Stoddard, R.W., and Stutzke, M. 1996. Software reliability models: An approach to early reliability prediction, In Proc. of 7th International Symposium on Software Reliability Engineering, pp. 132–141.
Takagi, Y., Tanaka, T., Niihara, N., Sakamoto, K., Kusumoto, S., and Kikuno, T. 1995. Analysis of review’s effectiveness based on software metrics, In Proc. of 5th International Symposium on Software Reliability Engineering, pp. 34–39.
Tanaka, T., Sakamoto, K., Kusumoto, S., Matsumoto, K., and Kikuno, T. 1995. Improvement of software process by process description and benefit estimation, In Proc. of 17th International Conference on Software Engineering, pp. 123–132.
Yokoyama, Y. and Kodaira, M. 1998. Software cost and quality analysis by statistical approaches, In Proc. 20th International Conference on Software Engineering, pp. 465–467.
Author information
Authors and Affiliations
Corresponding author
Additional information
Sousuke Amasakireceived the B.E. degree in Information and Computer Sciences from Okayama Prefectural University, Japan, in 2000 and the M.E. degree in Information and Computer Sciences from Graduate School of Information Science and Technology, Osaka University, Japan, in 2003. He has been in Ph.D. course of Graduate School of Information Science and Technology at Osaka University. His interests include the software process and the software quality assurance technique. He is a student member of IEEE and ACM.
Takashi Yoshitomireceived the B.E. degree in Information and Computer Sciences from Osaka University, Japan, in 2002. He has been working for Hitachi Software Engineering Co., Ltd.
Osamu Mizunoreceived the B.E., M.E., and Ph.D. degrees in Information and Computer Sciences from Osaka University, Japan, in 1996, 1998, and 2001, respectively. He is an Assistant Professor of the Graduate School of Information Science and Technology at Osaka University. His research interests include the improvement technique of the software process and the software risk management technique. He is a member of IEEE.
Yasunari Takagireceived the B.E. degree in Information and Computer Science, from Nagoya Institute of Technology, Japan, in 1985. He has been working for OMRON Corporation. He has been also in Ph.D. course of Graduate School of Information Science and Technology at Osaka University since 2002.
Tohru Kikunoreceived the B.E., M.Sc., and Ph.D. degrees in Electrical Engineering from Osaka University, Japan, in 1970, 1972, and 1975, respectively. He joined Hiroshima University from 1975 to 1987. Since 1990, he has been a Professor of the Department of Information and Computer Sciences at Osaka University. His research interests include the analysis and design of fault-tolerant systems, the quantitative evaluation of software development processes, and the design of procedures for testing communication protocols. He is a member of IEEE and ACM.
Rights and permissions
About this article
Cite this article
Amasaki, S., Yoshitomi, T., Mizuno, O. et al. A New Challenge for Applying Time Series Metrics Data to Software Quality Estimation. Software Qual J 13, 177–193 (2005). https://doi.org/10.1007/s11219-005-6216-8
Issue Date:
DOI: https://doi.org/10.1007/s11219-005-6216-8