Skip to main content

Advertisement

Log in

Research on key technologies of edge cache in virtual data space across WAN

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

The authors of this paper have previously proposed the global virtual data space system (GVDS) to aggregate the scattered and autonomous storage resources in China’s national supercomputer grid (National Supercomputing Center in Guangzhou, National Supercomputing Center in Jinan, National Supercomputing Center in Changsha, Shanghai Supercomputing Center, and Computer Network Information Center in Chinese Academy of Sciences) into a storage system that spans the wide area network (WAN), which realizes the unified management of global storage resources in China. At present, the GVDS has been successfully deployed in the China National Grid environment. However, when accessing and sharing remote data in the WAN, the GVDS will cause redundant transmission of data and waste a lot of network bandwidth resources. In this paper, we propose an edge cache system as a supplementary system of the GVDS to improve the performance of upper-level applications accessing and sharing remote data. Specifically, we first designs the architecture of the edge cache system, and then study the key technologies of this architecture: the edge cache index mechanism based on double-layer hashing, the edge cache replacement strategy based on the GDSF algorithm, the request routing based on consistent hashing method, and the cluster member maintenance method based on the SWIM protocol. The experimental results show that the edge cache system has successfully implemented the relevant operation functions (read, write, deletion, modification, etc.) and is compatible with the POSIX interface in terms of function. Further, it can greatly reduce the amount of data transmission and increase the data access bandwidth when the accessed file is located at the edge cache system in terms of performance, i.e., its performance is close to the performance of the network file system in the local area network (LAN).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Ashton K. That ‘internet of things’ thing. RFID Journal, 2009, 22(7): 97–114

    Google Scholar 

  2. Liu L M, Wang B. Research of an end-to-end transfer mechanism for big data in CMAGrid environment. Computing Technology and Automation, 2014, 33(1): 122–126

    Google Scholar 

  3. Wang B, Zong X, Tian H. Design and establishment of a nationwide meteorological computational grid. Journal of Applied Meteorological Science, 2010, 21(5): 632–640

    Google Scholar 

  4. Li S, Xu L D, Zhao S. The internet of things: a survey. Information Systems Frontiers, 2015, 17(2): 243–259

    Article  Google Scholar 

  5. Dilley J, Maggs B, Parikh J, Prokop H, Sitaraman R, Weihl B. Globally distributed content delivery. IEEE Internet Computing, 2002, 6(5): 50–58

    Article  Google Scholar 

  6. Satyanarayanan M. The emergence of edge computing. Computer, 2017, 50(1): 30–39

    Article  Google Scholar 

  7. Su Z, Dai M, Xu Q, Li R, Fu S. Q-learning-based spectrum access for content delivery in mobile networks. IEEE Transactions on Cognitive Communications and Networking, 2020, 6(1): 35–47

    Article  Google Scholar 

  8. Ramaswamy L, Liu L, Iyengar A. Scalable delivery of dynamic content using a cooperative edge cache grid. IEEE Transactions on Knowledge and Data Engineering, 2007, 19(5): 614–630

    Article  Google Scholar 

  9. Zhao J. The case for VM-based cloudlets in mobile computing. See Docin.Com/P-1950150101 website, 2010

  10. Wilkinson S, Boshevski T, Brandoff J, Buterin V. Storj a peer-to-peer cloud storage network. See Citeseerx.ist.psu.edu/viewdoc/download; jsessionid=D732CB9C4DBDF36BFBCA7F5821A593AC?doi=10.1.1.693.785&rep=rep1&type=website, 2014

  11. Chen B, Yang C, Wang G. High-throughput opportunistic cooperative device-to-device communications with caching. IEEE Transactions on Vehicular Technology, 2017, 66(8): 7527–7539

    Article  Google Scholar 

  12. Tan H, Jiang H C, Han Z, Liu L, Zhao Q. Camul: online caching on multiple caches with relaying and bypassing. In: Proceedings of 2019 IEEE Conference on Computer Communications. 2019

  13. Headquarters A. Cisco wide area application services configuration guide. See Cisco.Com/C/En/Us/Support/Routers/Wide-Area-Applicati-on-Services-Waas-Software/Products-Configuration-Guides-List

  14. Berg B, Berger D S, McAllister S, Grosof I, Gunasekar S, Lu J, Uhlar M, Carrig J, Beckmann N, Harchol-Balter M, Ganger G R. The Cachelib caching engine: design and experiences at scale. In: Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation. 2020, 753–768

  15. Cherkasova L. Improving WWW proxies performance with greedy-dual-size-frequency caching policy. See hpl.hp.com/techreports/98/HPL-98–69R1 website, 1998

  16. Karger D, Lehman E, Leighton T, Panigrahy R, Levine M, Lewin D. Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web. In: Proceedings of the Twenty-Ninth Annual ACM Symposium on Theory of Computing. 1997, 654–663

  17. Das A, Gupta I, Motivala A. SWIM: scalable weakly-consistent infection-style process group membership protocol. In: Proceedings International Conference on Dependable Systems and Networks. 2002, 303–312

  18. Birman K. The promise, and limitations, of gossip protocols. ACM SIGOPS Operating Systems Review, 2007, 41(5): 8–13

    Article  Google Scholar 

Download references

Acknowledgements

The work described in this paper was supported by the National Key Research and Development Program of China (2018YFB0203901), the National Natural Science Foundation of China (Grant No. 61772053), the Hebei Youth Talents Support Project (BJ2019008), the Natural Science Foundation of Hebei Province (F2020204003).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zhisheng Huo.

Additional information

Jiantong Huo is a PhD candidate in the school of Computer Science and Technology, Beihang University, China. He received MS degree of College of Computer Science from Beihang University, China in 2012. His research focuses on distributed storage system, system security and computer network.

Yaowen Xu is a PhD candidate in the College of Computer Science and Technology, Zhejiang University, China. He received MS degree of College of Computer Science from Beihang University, China in 2020. His research focuses on big storage system.

Zhisheng Huo is an Assistant Professor of high performance computing center, College of Software, Beihang University, China. His research interests include big data storage and distributed storage system.

Limin Xiao is a Professor in the school of Computer Science and Technology, Beihang University, China. His main research areas are computer architecture, computer system software, high performance computing, virtualization and cloud computing.

Zhenxue He is currently a Full Associate Professor with Agricultural University of Hebei, China. His research interests include low power integrated circuit design and optimization, multiplevalued logic circuits and intelligent algorithm. He is a member of China Computer Federation.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huo, J., Xu, Y., Huo, Z. et al. Research on key technologies of edge cache in virtual data space across WAN. Front. Comput. Sci. 17, 171102 (2023). https://doi.org/10.1007/s11704-022-1176-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-022-1176-8

Keywords