Abstract
The authors of this paper have previously proposed the global virtual data space system (GVDS) to aggregate the scattered and autonomous storage resources in China’s national supercomputer grid (National Supercomputing Center in Guangzhou, National Supercomputing Center in Jinan, National Supercomputing Center in Changsha, Shanghai Supercomputing Center, and Computer Network Information Center in Chinese Academy of Sciences) into a storage system that spans the wide area network (WAN), which realizes the unified management of global storage resources in China. At present, the GVDS has been successfully deployed in the China National Grid environment. However, when accessing and sharing remote data in the WAN, the GVDS will cause redundant transmission of data and waste a lot of network bandwidth resources. In this paper, we propose an edge cache system as a supplementary system of the GVDS to improve the performance of upper-level applications accessing and sharing remote data. Specifically, we first designs the architecture of the edge cache system, and then study the key technologies of this architecture: the edge cache index mechanism based on double-layer hashing, the edge cache replacement strategy based on the GDSF algorithm, the request routing based on consistent hashing method, and the cluster member maintenance method based on the SWIM protocol. The experimental results show that the edge cache system has successfully implemented the relevant operation functions (read, write, deletion, modification, etc.) and is compatible with the POSIX interface in terms of function. Further, it can greatly reduce the amount of data transmission and increase the data access bandwidth when the accessed file is located at the edge cache system in terms of performance, i.e., its performance is close to the performance of the network file system in the local area network (LAN).
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Ashton K. That ‘internet of things’ thing. RFID Journal, 2009, 22(7): 97–114
Liu L M, Wang B. Research of an end-to-end transfer mechanism for big data in CMAGrid environment. Computing Technology and Automation, 2014, 33(1): 122–126
Wang B, Zong X, Tian H. Design and establishment of a nationwide meteorological computational grid. Journal of Applied Meteorological Science, 2010, 21(5): 632–640
Li S, Xu L D, Zhao S. The internet of things: a survey. Information Systems Frontiers, 2015, 17(2): 243–259
Dilley J, Maggs B, Parikh J, Prokop H, Sitaraman R, Weihl B. Globally distributed content delivery. IEEE Internet Computing, 2002, 6(5): 50–58
Satyanarayanan M. The emergence of edge computing. Computer, 2017, 50(1): 30–39
Su Z, Dai M, Xu Q, Li R, Fu S. Q-learning-based spectrum access for content delivery in mobile networks. IEEE Transactions on Cognitive Communications and Networking, 2020, 6(1): 35–47
Ramaswamy L, Liu L, Iyengar A. Scalable delivery of dynamic content using a cooperative edge cache grid. IEEE Transactions on Knowledge and Data Engineering, 2007, 19(5): 614–630
Zhao J. The case for VM-based cloudlets in mobile computing. See Docin.Com/P-1950150101 website, 2010
Wilkinson S, Boshevski T, Brandoff J, Buterin V. Storj a peer-to-peer cloud storage network. See Citeseerx.ist.psu.edu/viewdoc/download; jsessionid=D732CB9C4DBDF36BFBCA7F5821A593AC?doi=10.1.1.693.785&rep=rep1&type=website, 2014
Chen B, Yang C, Wang G. High-throughput opportunistic cooperative device-to-device communications with caching. IEEE Transactions on Vehicular Technology, 2017, 66(8): 7527–7539
Tan H, Jiang H C, Han Z, Liu L, Zhao Q. Camul: online caching on multiple caches with relaying and bypassing. In: Proceedings of 2019 IEEE Conference on Computer Communications. 2019
Headquarters A. Cisco wide area application services configuration guide. See Cisco.Com/C/En/Us/Support/Routers/Wide-Area-Applicati-on-Services-Waas-Software/Products-Configuration-Guides-List
Berg B, Berger D S, McAllister S, Grosof I, Gunasekar S, Lu J, Uhlar M, Carrig J, Beckmann N, Harchol-Balter M, Ganger G R. The Cachelib caching engine: design and experiences at scale. In: Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation. 2020, 753–768
Cherkasova L. Improving WWW proxies performance with greedy-dual-size-frequency caching policy. See hpl.hp.com/techreports/98/HPL-98–69R1 website, 1998
Karger D, Lehman E, Leighton T, Panigrahy R, Levine M, Lewin D. Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web. In: Proceedings of the Twenty-Ninth Annual ACM Symposium on Theory of Computing. 1997, 654–663
Das A, Gupta I, Motivala A. SWIM: scalable weakly-consistent infection-style process group membership protocol. In: Proceedings International Conference on Dependable Systems and Networks. 2002, 303–312
Birman K. The promise, and limitations, of gossip protocols. ACM SIGOPS Operating Systems Review, 2007, 41(5): 8–13
Acknowledgements
The work described in this paper was supported by the National Key Research and Development Program of China (2018YFB0203901), the National Natural Science Foundation of China (Grant No. 61772053), the Hebei Youth Talents Support Project (BJ2019008), the Natural Science Foundation of Hebei Province (F2020204003).
Author information
Authors and Affiliations
Corresponding author
Additional information
Jiantong Huo is a PhD candidate in the school of Computer Science and Technology, Beihang University, China. He received MS degree of College of Computer Science from Beihang University, China in 2012. His research focuses on distributed storage system, system security and computer network.
Yaowen Xu is a PhD candidate in the College of Computer Science and Technology, Zhejiang University, China. He received MS degree of College of Computer Science from Beihang University, China in 2020. His research focuses on big storage system.
Zhisheng Huo is an Assistant Professor of high performance computing center, College of Software, Beihang University, China. His research interests include big data storage and distributed storage system.
Limin Xiao is a Professor in the school of Computer Science and Technology, Beihang University, China. His main research areas are computer architecture, computer system software, high performance computing, virtualization and cloud computing.
Zhenxue He is currently a Full Associate Professor with Agricultural University of Hebei, China. His research interests include low power integrated circuit design and optimization, multiplevalued logic circuits and intelligent algorithm. He is a member of China Computer Federation.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Huo, J., Xu, Y., Huo, Z. et al. Research on key technologies of edge cache in virtual data space across WAN. Front. Comput. Sci. 17, 171102 (2023). https://doi.org/10.1007/s11704-022-1176-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11704-022-1176-8