Abstract
We propose a storage workload model able to process discrete time series incrementally, continually updating its parameters with the availability of new data. More specifically, a Hidden Markov Model (HMM) with an adaptive Baum-Welch algorithm is trained on two raw traces: a NetApp network trace consisting of timestamped I/O commands and a Microsoft trace also with timestamped entries containing reads and writes. Each of these traces is analyzed statistically and HMM parameters are inferred, from which a fluid input model with rates modulated by a Markov chain is derived. We generate new data traces using this Markovian fluid, workload model. To validate our parsimonious model, we compare statistics of the raw and generated traces and use the Viterbi algorithm to produce representative sequences of the hidden states. The incremental model is measured against both the standard model (parameterized on the whole dataset) and the raw data trace.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Baum, L.E., Petrie, T.: Stastical Inference for Probabilistic Functions of Finite Markov Chains. The Annals of Mathematical Statistics 37, 1554–1563 (1966)
Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. The Annals of Mathematical Statistics 41, 164–171 (1970)
Viterbi, A.J.: Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory 13, 260–269 (1967)
Krough, A., Brown, M., Mian, S., Sjolander, K., Haussler, D.: Hidden Markov Models in Computational Biology. Journal of Molecular Biology, 1501–1531 (1994)
Burge, C., Karlin, S.: Prediction of complete gene structures in human genomic DNA. Journal of Molecular Biology, 78–94 (1997)
Ashraf, J., Iqbal, N., Khattak, N.S., Zaidi, A.M.: Speaker Independent Urdu Speech Recognition Using HMM (2010)
Harrison, P.G., Harrison, S.K., Patel, N.M., Zertal, S.: Storage Workload Modelling by Hidden Markov Models: Application to Flash Memory. Performance Evaluation 69, 17–40 (2012)
Florez-Larrahondo, G., Bridges, S., Hansen, E.A.: Incremental Estimation of Discrete Hidden Markov Models on a New Backward Procedure (2005)
Rabiner, L.R., Juang, B.H.: An Introduction to Hidden Markov Models. IEEE ASSP Magazine 3, 4–16 (1986)
Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. IEEE 77, 257–286 (1989)
Chis, T.: Hidden Markov Models: Applications to Flash Memory Data and Hospital Arrival Times (2011)
Zhang, X., Riska, A., Riedel, E.: Characterization of the E-commerce Storage Subsystem Workload. In: Proceedings of Quantitative Evaluation of SysTems (2008)
RUBiS Implementation: http://rubis.objectweb.org/
On-Line Transaction Processing (OLTP) Benchmark: http://www.tpc.org/tpcc
Kumas, Z., Keeton, K., Becker-Szendy, R.: I/O Workload Characterization. In: CAECW 2001, before HPCA-7 (2001)
Transactional Web e-Commerce Benchmark: http://www.tpc.org/tpcw
Chis, T., Harrison, P.G.: Incremental HMM with an improved Baum-Welch Algorithm. In: Proceedings of Imperial College Computing Student Workshop (2012)
Stenger, B., Ramesh, V., Paragois, N., Coetzee, F., Buhmann, J.M.: Topology free Hidden Markov Models: Application to background modeling. In: Proceedings of the International Conference on Computer Vision, pp. 297–301 (2001)
Keaton, K., Veitch, A., Obal, D., Wilkes, J.: I/O characterization of commercial workloads. In: CAECW 2000 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chis, T., Harrison, P.G. (2013). iSWoM: The Incremental Storage Workload Model Based on Hidden Markov Models. In: Dudin, A., De Turck, K. (eds) Analytical and Stochastic Modeling Techniques and Applications. ASMTA 2013. Lecture Notes in Computer Science, vol 7984. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39408-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-39408-9_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39407-2
Online ISBN: 978-3-642-39408-9
eBook Packages: Computer ScienceComputer Science (R0)