Abstract
In order to enhance scalability of Blue Whale Cluster File System (BWFS), this paper analyses the bottleneck of scalability of BWFS in non-linear editing application, and presents a solution that clients employ local disk storage as cache on clients. We called BW-DCache (BWFS Cache based Disk medium). Moreover, we carefully choose the client cache strategy and replacement algorithm according to data access pattern. By this, clients can get as much data from local cache as possible, avoid random accesses to storage servers and efficiently reduce the load of storage servers. Therefore, the scalability of the whole system is significantly enhanced. The experiment result shows that our solution can enhance the scalability of the Blue Whale System by 80% at most. Moreover, we adopt some optimizations to improve system performance further.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Yang, D., Huang, H., Zhang, J., Xu, L.: BWFS: A Distributed File System with Large Capacity, High Throughput and High Scalability. Journal of Computer Research and Development? 42(6), 1028–1033 (2005)
Howard, J.H.: An Overview of the Andrew File System, CMU-ITC-062, http://reports-archive.adm.cs.cmu.edu/itc85.html
Tanenbaum, A.S.: Modern Operating Systems. Prentice-Hall, Englewood Cliffs (1992)
Howard, J.H., Kazar, M.L., et al.: Scale and performance in distributed file system. ACM Transactions on Computer Systems 6(1), 51–81 (1988)
He, X., Zhang, M., Yang, Q.: STICS: SCSI-to-IP cache for storage area networks. Parallel and Distributed Computing 64(9), 1069–1085 (2004)
Sivathanu, G., Zadok, E.: A Versatile Persistent Caching Framework for File Systems, Stony Brook University, Technical Report FSL-05-05 (2005)
Hensbergen, E.V., Zhao, M.: Dynamic policy disk caching for storage networking. Technical Report RC24123, IBM Research Division Austin Research Laboratory (2006)
Dahlin, M., et al.: Cooperative Caching: Using Remote Client Memory to Improve File System Performance. In: Proceeding of the First Symposium on Operating Systems Design and Implementation, pp. 267–280 (1994)
Sarkar, P., Hartman, J.H.: Hint-based cooperative caching. ACM Transactions on Computer Systems 18(4), 387–419 (2000)
Denning, P.J., Schwartz, S.C.: Properties of the working set model. Communications of the ACM 15(3) (1972)
Smith, A.J.: Cache Memories. ACM Computing Surveys 14(3), 473–530 (1982)
Zhou, Y., Philbin, J.F., Li, K.: The Multi-Queue Replacement Algorithm for Second Level Buffer Caches. In: Proc. of 2001 Annual USENIX Technical Conference (2001)
Johnson, T., Shasha, D.: 2Q: A Low Overhead High Performance Buffer Management Replacement Algorithm. In: Proceedings of the 20th International Conference on VLDB, pp. 439–450 (1994)
Megiddo, N., Modha, D.S.: ARC: A self-tuning, low overhead replacement cache. In: Proc. 2nd USENIX Conference on File and Storage Technologies (FAST 2003), San Franciso, CA, pp. 115–130 (2003)
Sun Microsystems, Inc. NFS: Network File System Protocol Specification (1989)
Callaghan, B., et al.: Sun Microsystems, Inc. NFS Version 3 Protocol Specification (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Si, C., Meng, X., Xu, L. (2010). BW-DCache: An Inexpensive, Effective and Reliable Cache Solution in a SAN File System. In: Mewhort, D.J.K., Cann, N.M., Slater, G.W., Naughton, T.J. (eds) High Performance Computing Systems and Applications. HPCS 2009. Lecture Notes in Computer Science, vol 5976. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12659-8_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-12659-8_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12658-1
Online ISBN: 978-3-642-12659-8
eBook Packages: Computer ScienceComputer Science (R0)