Abstract
The ever-growing demand for high performance computation calls for progressively larger parallel distributed file systems to match their requirement. These file systems can achieve high performance for large I/O operations through distributing load across numerous data servers. However, they fail to provide quality service for applications pertaining to small files. In this paper, we propose a delegable metadata service (DMS) for hiding latency of metadata accesses and optimizing small-file performance. In addition, four techniques have been designed to maintain consistency and efficiency in DMS: pre-allocate serial metahandles, directory-based metadata replacement, packing transaction operations and fine-grained lock revocation. These schemes have been employed in Cappella parallel distributed file system, and various experiments complying with industrial standards have been conducted for evaluation of its efficiency. The results show that our design has achieved significant improvement in performance of both metadata operations and small-file access. Moreover, this scheme is widely applicable for integration within many other distributed file systems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Devulapalli A, Wyckoff P. File creation strategies in a distributed metadata file system. In: Proceedings of the 21st International Parallel and Distributed Processing Symposium, Long Beach, 2007. 1–10
The Luster File Systems. version 1.8.3. (FL): Distributed File Systems. 2008
Ghemawat S, Gobioff H, Leung S. The Google File System. In: Proceedings of the 19th ACM Symposium on Operating Systems Principles, New York, 2003. 29–43
Roselli D, Lorch J R, Anderson T E. A comparison of file system workloads. In: Proceedings of the USENIX Annual Technical Conference, San Diego, 2000. 41–54
Beave D, Kumar S, Li H C, et al. Finding a needle in Haystack: Fackbook’s photo storage. In: Proceedings of the 9th Symposium on Operating Systems Design and Implementation, Vancouver, 2010. 47–60
Pawlowski B, Juszczak C, Staubach P, et al. NFS version 3: design and implementation. In: Proceedings of the Summer USENIX Conference, Boston, 1994. 137–152
Carns P H, Ligon W B, Ross R B, et al. PVFS: a parallel file system for Linux clusters. In: Proceedings of the 4th Annual Linux Showcase and Conference, Atlanta, 2000. 317–327
Zhang Q, Feng D, Wang F. Metadata performance optimization in Distributed File System. In: Proceedings of IEEE/ACIS 11th International Conference on Computer and Information Science, Shanghai, 2012. 476–481
Carns P, Lang S, Ross R, et al. Small-file access in parallel file systems. In: Proceedings of the 23rd IEEE IPDPS, Rome, 2009. 1–11
Xia P, Feng D, Jiang H, et al. FARMER: a novel approach to file access correlation mining and evaluation reference model for optimizing peta-scale file system performance. In: Proceedings of the 17th International Symposium on HPDC, Boston, 2008. 185–196
Leung A W, Pasupathy S, Goodson G, et al. Measurement and analysis of large-scale network file system workloads. In: Proceedings of the USENIX Annual Technical Conference, Boston, 2008. 213–226
Welch B, Unangst M, Abbasi Z, et al. Scalable performance of the panasas parallel file system. In: Proceedings of the 6th USENIX Conference on FAST, San Jose, 2008. 17–33
Leung A W, Shao M L, Bisson T, et al. Spyglass: fast, scalable metadata search for large-scale storage systems. In: Proceedings of the 7th Conference on FAST, San Francisco, 2009. 153–166
Wang J, Feng D, Wang F, et al. MHS: a distributed metadata management strategy. J Syst Softw, 2009, 82: 2004–2011
Yu L H, Chen G, Wang W, et al. MSFSS: a storage system for mass small files. In: Proceedings of the 11th International Conference on Computer Supported Cooperative Work in Design, Melbourne, 2007. 1087–1092
Sinnamohideen S, Sambasivan R R, Hendricks J. A transparently-scalable metadata service for the Ursa Minor storage system. In: Proceedings of the USENIX ATC, Boston, 2010. 153–166
Xiong J, Hu Y M, Li G J, et al. Metadata distribution and consistency techniques for large-scale cluster file systems. IEEE Trans Parall Distrib Syst, 2011, 22: 803–816
Katcher J. Postmark: a New File System Benchmark. Network Appliance Technical Report TR3022, 1997
Zoe S, Kostas M, Manolis M, et al. A comparative experimental study of parallel file systems for large-scale data processing. In: Proceedings of the 1st USENIX Workshop on Large-Scale Computing, Berkeley, 2008
Patil S, Gibson G. Scale and concurrency of GIGA+: file system directories with millions of files. In: Proceedings of the 9th USENIX Conference on File and Storage Technologies, San Jose, 2011. 177–190
Xing J, Xiong J, Sun N H, et al. Adaptive and scalable metadata management to support a trillion files. In: Proceedings of the ACM/IEEE Conference on High Performance Computing, Portland, 2009. 1–11
Meshram V, Ouyang X Y, Panda D K. Minimizing Lookup RPCs in Lustre File System Using Metadata Delegation at Client Side. Department of Compute Science and Engineering, Ohio State University Technical Report TR20, 2011
Hendricks J, Sambasivan R R, Sinhamohideen S, et al. Improving small file performance in object-based storage. In: Proceedings of IEEE International Conference on Services Computing, Miami, 2010. 65–72
Kuhn M, Kunkel J M, Ludwig T. Dynamic file system semantics to enable metadata optimizations in PVFS. Concurr Comput-Pract Exper, 2009, 21: 1775–1788
Li X Q, Dong B, Xiao L M, et al. Adaptive tradeoff in metadata-based small file optimization for a cluster file system. Int J Numer Anal Model, 2012, 9: 289–303
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, Q., Feng, D., Wang, F. et al. Mlock: building delegable metadata service for the parallel file systems. Sci. China Inf. Sci. 58, 1–14 (2015). https://doi.org/10.1007/s11432-014-5194-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11432-014-5194-5