Skip to main content
SpringerLink
Log in
Book cover

International Conference on Web Information Systems Engineering

WISE 2021: Web Information Systems Engineering – WISE 2021 pp 87–100Cite as

A Low-Latency Metadata Service for Geo-Distributed File Systems

  • Conference paper
  • First Online:

  • 1425 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13080))

Abstract

Geo-distributed file systems have been widely used by web services. An increasing number of time-critical web applications have been deployed on the cloud across geographical regions. In this circumstance, intolerant service latency will occur when user accesses remote servers. In view of this challenge, We design a metadata service which aims at reducing the service invocation latency. This low-latency metadata service is named LoLaMS. Taking advantage of the latency-aware dynamic subtree partition and migration, LoLaMS is capable to handle more metadata service invocations in the nearby metadata server. On account of that, the expected latency could be satisfied by LoLaMS. We implemented the LoLaMS and deployed it in a real-word cloud environment across different regions. The experimental results show that LoLaMS reduces the network latency effectively while ensuring high metadata consistency.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Alves, M.P., Delicato, F.C., Santos, I.L., Pires, P.F.: LW-CoEdge: a lightweight virtualization model and collaboration process for edge computing. World Wide Web 23(2), 1127–1175 (2020)

    Article  Google Scholar 

  2. Benet, J.: IPFS-content addressed, versioned, p2p file system. arXiv preprint arXiv:1407.3561 (2014)

  3. Braam, P.: The lustre storage architecture. CoRR abs/1903.01955 (2019). http://arxiv.org/abs/1903.01955

  4. Confais, B., Lebre, A., Parrein, B.: Performance analysis of object store systems in a fog and edge computing infrastructure. In: Hameurlain, A., Küng, J., Wagner, R., Akbarinia, R., Pacitti, E. (eds.) Transactions on Large-Scale Data- and Knowledge-Centered Systems XXXIII. LNCS, vol. 10430, pp. 40–79. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-662-55696-2_2

    Chapter  Google Scholar 

  5. Facebook: Rocksdb. http://rocksdb.org/

  6. Ghemawat, S., Gobioff, H., Leung, S.T.: The google file system. In: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, pp. 29–43 (2003)

    Google Scholar 

  7. Hunt, P., Konar, M., Junqueira, F.P., Reed, B.: Zookeeper: wait-free coordination for internet-scale systems. In: USENIX Annual Technical Conference, vol. 8 (2010)

    Google Scholar 

  8. Kubiatowicz, J., et al.: Oceanstore: an architecture for global-scale persistent storage. ACM SIGOPS Oper. Syst. Rev. 34(5), 190–201 (2000)

    Article  Google Scholar 

  9. Oh, K., Qin, N., Chandra, A., Weissman, J.: Wiera: policy-driven multi-tiered geo-distributed cloud storage system. IEEE Trans. Parallel Distrib. Syst. 31(2), 294–305 (2019)

    Article  Google Scholar 

  10. Qi, L., Chen, Y., Yuan, Y., Fu, S., Zhang, X., Xu, X.: A QoS-aware virtual machine scheduling method for energy conservation in cloud-based cyber-physical systems. World Wide Web 23(2), 1275–1297 (2020)

    Article  Google Scholar 

  11. Ren, K., Li, D., Abadi, D.J.: Slog: serializable, low-latency, geo-replicated transactions. Proc. VLDB Endowment 12(11), 1747–1761 (2019)

    Article  Google Scholar 

  12. Roselli, D.S., Lorch, J.R., Anderson, T.E., et al.: A comparison of file system workloads. In: USENIX Annual Technical Conference, General Track, pp. 41–54 (2000)

    Google Scholar 

  13. Ross, R.B., Thakur, R., et al.: PVFS: a parallel file system for linux clusters. In: Proceedings of the 4th Annual Linux Showcase and Conference, pp. 391–430 (2000)

    Google Scholar 

  14. Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–10. IEEE (2010)

    Google Scholar 

  15. Singh, H.J., Bawa, S.: Scalable metadata management techniques for ultra-large distributed storage systems-a systematic review. ACM Comput. Surv. (CSUR) 51(4), 1–37 (2018)

    Article  Google Scholar 

  16. Uluyol, M., Huang, A., Goel, A., Chowdhury, M., Madhyastha, H.V.: Near-optimal latency versus cost tradeoffs in geo-distributed storage. In: 17th \(\{\)USENIX\(\}\) Symposium on Networked Systems Design and Implementation (\(\{\)NSDI\(\}\) 20), pp. 157–180 (2020)

    Google Scholar 

  17. Weil, S.A., Brandt, S.A., Miller, E.L., Long, D.D., Maltzahn, C.: Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation, pp. 307–320 (2006)

    Google Scholar 

  18. Weil, S.A., Pollack, K.T., Brandt, S.A., Miller, E.L.: Dynamic metadata management for petabyte-scale file systems. In: SC’04: Proceedings of the 2004 ACM/IEEE Conference on Supercomputing, p. 4. IEEE (2004)

    Google Scholar 

  19. Xu, X., et al.: Secure service offloading for internet of vehicles in SDN-enabled mobile edge computing. IEEE Trans. Intell. Transp. Syst. 22(6), 3720–3729 (2020)

    Article  Google Scholar 

  20. Xu, X., Zhang, X., Gao, H., Xue, Y., Qi, L., Dou, W.: Become: blockchain-enabled computation offloading for IoT in mobile edge computing. IEEE Trans. Ind. Inf. 16(6), 4187–4195 (2019)

    Article  Google Scholar 

  21. Yang, S., Wang, X., Wang, X., An, L., Zhang, G.: High-performance docker integration scheme based on openstack. World Wide Web 23(4), 2593–2632 (2020)

    Article  Google Scholar 

  22. Yu, H., Zhang, F., Wu, Y.: Granary: a sharing oriented distributed storage system. Future Gen. Comput. Syst. 38, 47–60 (2014)

    Article  Google Scholar 

  23. Yu, J., Wu, W., Yang, D., Huang, N., et al.: MRFS: a distributed files system with geo-replicated metadata. In: Sun, X. (ed.) ICA3PP 2014, Part II. LNCS, vol. 8631, pp. 273–285. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11194-0_21

    Chapter  Google Scholar 

  24. Zaker, F.: Online Shopping Store - Web Server Logs (2019). https://doi.org/10.7910/DVN/3QBYB5

  25. Zhou, J., Chen, Y., Wang, W., He, S., Meng, D.: A highly reliable metadata service for large-scale distributed file systems. IEEE Trans. Parallel Distrib. Syst. 31(2), 374–392 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China under Grant No. 2020YFB1707601, the Key Research and Development Program of Jiangsu Province under Grant No. BE2019104. Dr Xuyun Zhang is the recipient of an ARC DECRA (project No. DE210101458) funded by the Australian Government.

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

    Chuangwei Lin, Bowen Liu, Wei Zhou, Yueyue Xu & Wanchun Dou

  2. Department of Computing, Macquarie University, Sydney, Australia

    Xuyun Zhang

Authors
  1. Chuangwei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bowen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yueyue Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuyun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wanchun Dou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xuyun Zhang or Wanchun Dou .

Editor information

Editors and Affiliations

  1. School of Computer Science and Engineering, University of New South Wales, Sydney, Australia

    Wenjie Zhang

  2. Peking University, Beijing, China

    Lei Zou

  3. Zayed University, Dubai, United Arab Emirates

    Zakaria Maamar

  4. Swinburne University of Technology, Melbourne, VIC, Australia

    Lu Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lin, C., Liu, B., Zhou, W., Xu, Y., Zhang, X., Dou, W. (2021). A Low-Latency Metadata Service for Geo-Distributed File Systems. In: Zhang, W., Zou, L., Maamar, Z., Chen, L. (eds) Web Information Systems Engineering – WISE 2021. WISE 2021. Lecture Notes in Computer Science(), vol 13080. Springer, Cham. https://doi.org/10.1007/978-3-030-90888-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-90888-1_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-90887-4

  • Online ISBN: 978-3-030-90888-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation