Abstract
Understanding and predicting disk failures are essential for both disk vendors and users to manufacture more reliable disk drives and build more reliable storage systems, in order to avoid service downtime and possible data loss. Predicting disk failure from observable disk attributes, such as those provided by the Self-Monitoring and Reporting Technology (SMART) system, has been shown to be effective. In the paper, we treat SMART data as time series, and explore the prediction power by using HMM- and HSMM-based approaches. Our experimental results show that our prediction models outperform other models that do not capture the temporal relationship among attribute values over time. Using the best single attribute, our approach can achieve a detection rate of 46% at 0% false alarm. Combining the two best attributes, our approach can achieve a detection rate of 52% at 0% false alarm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Cole, G.: Estimating Drive reliability in Desktop Computers and consumer electronics systems. Tech. Rep., Seagate Technology Paper TP-338.1 (2000)
Pinheiro, E., Weber, W., Barroso, L.A.: Failure Trends in a Large Disk Drive Population. In: 5th USENIX Conference on File and Storage Technologies (FAST 2007), Berkeley, CA (2007)
Jiang, W., Hu, C., Zhou, Y., Kanevsky, A.: Are Disks the Dominant Contributor for Storage Failures? A Comprehensive Study of Storage Subsystem Failure Characteristics. ACM Transactions on Storage 4(3), Article 7 (2008)
Bairavasundaram, L.N., Goodson, G.R., Pasupathy, S., Schindler, J.: An Analysis of Latent Sector Errors in Disk Drives. SIGMETRICS Perform. Eval. Rev. 35(1), 289–300 (2007)
Hamerly, G., Elkan, C.: Bayesian Approaches to Failure Prediction for Disk Drives. In: 18th International Conference on Machine Learning (ICML 2001), Williamstown, MA (2001)
Murray, J.F., Hughes, G.F., Kreutz-Delgado, K.: Machine Learning Methods for Predicting Failures in Hard Drives: A Multiple-Instance Application. Journal of Machine Learning Research 6, 783–816 (2005)
Baum, L.E., Petrie, T.: Statistical Inference for Probabilistic Functions for Finite State Markov Chains. Annals of Mathematical Statistics 37, 1554–1563 (1966)
Ferguson, J.D.: Variable Duration Models for Speech. In: Symposium on the Application of Hidden Markov Models to Text and Speech, Princeton, New Jersey, pp. 143–179 (1980)
Rabiner, L.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. IEEE Transactions on Information Theory 77(2), 257–284 (1989)
Lehmann, E.L., D’Abrera, H.J.M.: Nonparametrics: Statistical Methods Based on Ranks. Prentice Hall, Upper Saddle River (1998)
Duin, R.P.W.: The Combining Classifier: to Train or not to Train? In: 16th international conference on Pattern Recognition, Quebec City, Canada, pp. 765–770 (2002)
Vapnik, V.: Statistical Learning Theory. John Wiley, New York (1998)
Salfner, F., Malek, M.: Using Hidden Semi-Markov Models for Effective Online Failure Prediction. In: 26th IEEE Symposium on Reliable Distributed Systems (SRDS 2007), Beijing, China, pp. 161–174 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhao, Y., Liu, X., Gan, S., Zheng, W. (2010). Predicting Disk Failures with HMM- and HSMM-Based Approaches. In: Perner, P. (eds) Advances in Data Mining. Applications and Theoretical Aspects. ICDM 2010. Lecture Notes in Computer Science(), vol 6171. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14400-4_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-14400-4_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-14399-1
Online ISBN: 978-3-642-14400-4
eBook Packages: Computer ScienceComputer Science (R0)