Abstract
In this paper, we present a system capable of improving the I/O performance in an automatic way. This system is able to learn the behavior of the applications running on top and find the best data placement in the disk in order to improve the I/O performance. This system is built by three independent modules. The first one is able to learn the behavior of a workload in order to be able to reproduce its behavior later on, without a new execution. The second module is a drive modeler that is able to learn how a storage drive works taking it as a “black box”. Finally, the third module generates a set of placement alternatives and uses the afore mentioned models to predict the performance each alternative will achieve. We tested the system with five benchmarks and the system was able to find better alternatives in most cases and improve the performance significantly (up to 225%). Most important, the performance predicted where always very accurate (less that 10% error).
This work was supported in part by a grant from FONACIT (Venezuela) which is gratefully acknowledged, by the Ministry of Science and Technology (Spain), and by FEDER funds of the European Union under grants TIC2001-0995-C02-01.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Hidrobo, F., Cortes, T.: Towards a Zero-Knowledge Model for Disk Drives. In: Proceedings of the AMS, Seattle, WA, USA. IEEE Computer Society Press, Los Alamitos (2003)
Smith, K.A., Seltzer, M.I.: File System Aging Increasing the Relevance of File System Benchmark. In: Proceedings of SIGMETRICS (1997)
Ruemmler, C., Wilkes, J.: An introduction to disk drive modeling. IEEE Computer 27, 17–28 (1994)
Ganger, G., Worthington, B., Patt, Y.: The DiskSim Simulation Environment (Version 2.0) (2004), http://www.ece.cmu.edu/~ganger/disksim/
Schindler, J., Ganger, G.R.: Automated Disk Drive Characterization. In: Proceedings of SIGMETRICS, Santa Clara, CA, USA, pp. 112–113. ACM Press, New York (2000)
Shriver, E., Merchant, A., Wilkes, J.: An analytic behavior model for disk drives with readahead caches and requests reordering. In: SIGMETRICS, Madison, Wisconsin, USA, pp. 182–191. ACM Press, New York (1998)
Thornock, N.C., Tu, X.H., kelly Flanagan, J.: A Stochastic Disk I/O Simulation Technique. In: Proceedings of Winter Simulation Conference, Atlanta, GA, USA, pp. 1079–1086. ACM Press, New York (1997)
Anderson, E.: Simple table-based modeling of storage devices. Technical Report HPL-SSP-2001-04, HP Laboratories (2001), http://www.hpl.hp.com/SSP/papers/
Kotz, D., Nieuwejaar, N.: Dynamic File-Access Characteristics of a Production Parallel Scientific Workload. In: Proceedings of Supercomputing, pp. 640–649. IEEE Computer Society Press, Washington (1994)
Kroeger, T.M., Long, D.D.: The Case for Efficient File Access Pattern Modeling. In: Proceedings of HotOS, AZ, USA, pp. 14–19. IEEE Computer Society, Los Alamitos (1999)
Ware, P.P., Page Jr., T.W., Nelson, B.L.: Modeling File-system Input Traces via a Two-level Arrival Process. In: Proceedings of the 28th Conference on Winter Simulation, Coronado, California, United States, pp. 1230–1237. ACM Press, New York (1996)
Ware, P.P., Page Jr., T.W.: Automatic Modeling of File System Workloads Using Two-Level Arrival Processes. ACM TOMACS 8, 305–330 (1998)
Ruemmler, C., Wilkes, J.: UNIX Disk Access Patterns. In: Proceedings of Winter USENIX Conference, San Diego, CA, USA, pp. 405–420 (1993)
Ganger, G.R.: Generating Representative Synthetic Workloads. An Unsolved Problem. In: Proceedings of 21st International Computer Measurement Group Conference, Nashville, TN, USA, pp. 1263–1269. Computer Measurement Group (1995)
Gómez, M.E., Santoja, V.: A new approach in the analysis and modeling of disk access pattern. In: Proceedings of ISPASS, Austin, Texas. IEEE Press, Los Alamitos (2000)
Madhyastha, T., Reed, D.: Input/Output Access Pattern Classification Using Hidden Markov Models. In: IOPADS, San Jose, CA, USA. ACM Press, New York (1997)
Oly, J.: Markov Model Prediction of I/O requests for Scientific Applications. Master’s project, University of Illinois at Urbana-Champaign (2000)
Vongsathorn, P., Carson, S.: A system for adaptive disk rearrangement. Software - Practice and Experience 20, 225–242 (1990)
Akyürek, S., Salem, K.: Adaptive Block Rearrangement. ACM TOCS 13, 89–121 (1995)
Alvarez, G., Borowsky, E., Go, S., Romer, T., Becker-Szendy, R., Golding, R., Merchant, A., Spasojevic, M., Veitch, A., Wilkes, J.: MINERVA: an automated resource provisioning tool for large-scale storage systems. TOCS 19, 483 (2001)
Anderson, E., Hobbs, M., Keeton, K., Spence, S., Uysal, M., Veitch, A.: Hippodrome: running circles around storage administration. In: Proceedings of FAST, Monterey, CA, USA, pp. 175–188. USENIX, Berkeley (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hidrobo, F., Cortes, T. (2004). Autonomic Storage System Based on Automatic Learning. In: Bougé, L., Prasanna, V.K. (eds) High Performance Computing - HiPC 2004. HiPC 2004. Lecture Notes in Computer Science, vol 3296. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30474-6_43
Download citation
DOI: https://doi.org/10.1007/978-3-540-30474-6_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24129-4
Online ISBN: 978-3-540-30474-6
eBook Packages: Computer ScienceComputer Science (R0)