Abstract
With the development of smart devices and social media, massive unstructured data is uploaded to distributed storage systems. Since the characteristics of multi-users and high concurrency the unstructured data accesses have, it brings new challenges to traditional distributed storage systems designed for large files. We propose a grouping strategy to analyze relevant data in access according to disk access logs in the real distributed storage systems environment. When any data in the group is accessed, the whole group is prefetched from disk to the cache. Firstly, we conduct statistical analysis on the access logs and propose a preliminary classification method to classify files in spatiotemporal locality. Secondly, a strength-priority tree structure relation model (SP-TSRM) is proposed to mine file group efficiently. Finally, experiments show that the proposed model can improve the cache hit rate significantly, thereby improving the read efficiency of distributed storage systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dong, B., Zheng, Q., Tian, F.: An optimized approach for storing and accessing small files on cloud storage. J. Netw. Comput. Appl. 35(6), 1847–1862 (2012)
Zhu, Y., Zhang, X., Zhao, R., Dong, X.: Data De-duplication on similar file detection. In: Innovative Mobile and Internet Services in Ubiquitous Computing, pp. 66–73. IEEE Press (2014)
Cui, Y., Lai, Z., Wang, X., Dai, N.: QuickSync: improving synchronization efficiency for mobile cloud storage services. IEEE Trans. Mob. Comput. 16, 3513–3526 (2017)
Dong, B., Qiu, J., Zheng, Q., Zhong, X., Li, J., Li, Y.: A novel approach to improving the efficiency of storing and accessing small files on hadoop: a case study by powerpoint files. In: Services Computing (SCC), pp. 65–72. IEEE Press (2010)
Bok, K., Lim, J., Oh, H., Yoo, J.: An efficient cache management scheme for accessing small files in distributed file systems. In: Big Data and Smart Computing (BigComp), pp. 151–155. IEEE Press (2017)
Lin, L., Li, X., Jiang, H., Zhu, Y., Tian, L., AMP: an affinity-based metadata prefetching scheme in large-scale distributed storage systems. In: Cluster Computing and the Grid, pp. 459–466. IEEE Press (2008)
Zhu, D., et al.: An access prefetching strategy for accessing small files based on swift. Procedia Comput. Sci. 131, 816–824 (2018)
Cherubini, G., Kim, Y., Lantz, M., Venkatesan, V.: Data prefetching for large tiered storage systems. In: Data Mining (ICDM), pp. 823–828. IEEE Press (2017)
Kroeger, T.M., Long, D.D., Mogul, J.C.: Exploring the bounds of web latency reduction from caching and prefetching. In: USENIX Symposium on Internet Technologies and Systems, pp. 13–22 (1997)
Wildani, A., Miller, E.L.: Can we group storage? Statistical techniques to identify predictive groupings in storage system accesses. ACM Trans. Storage (TOS) 12(2), 7–40 (2016)
Acknowledgements
Thanks to the students of HIT at Weihai. This work is supported by project under Grant no. 520613170002, SGSDWH00YXJS1700522, SGSDWH00YXJS1700270, the Fundamental Research Funds for the Central Universities (Grant No. HIT.NSRIF.201714), Weihai Science and Technology Development Program (2016DXGJMS15) and Key Research and Development Program in Shandong Provincial (2017GGX90103).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, D., Du, H., Cao, N., Qiao, X., Liu, Y. (2018). SP-TSRM: A Data Grouping Strategy in Distributed Storage System. In: Vaidya, J., Li, J. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2018. Lecture Notes in Computer Science(), vol 11334. Springer, Cham. https://doi.org/10.1007/978-3-030-05051-1_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-05051-1_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05050-4
Online ISBN: 978-3-030-05051-1
eBook Packages: Computer ScienceComputer Science (R0)