Skip to main content
SpringerLink
Log in

GHStore: A High Performance Global Hash Based Key-Value Store

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13245))

Included in the following conference series:

  • 3066 Accesses

Abstract

Log-Structured Merge tree (LSM-tree) has become the mainstream data structure of persistent key-value (KV) stores, but it suffers from serious write and read amplification. In update intensive workloads, repeated and useless compaction of outdated data makes the problem more serious. So we design an efficient global segmented hashmap to record the level of the latest KV pairs, and we present GHStore based on it, which is a key-value store that improves overall performance in write, read and range query simultaneously for update intensive workloads. A read operation of GHStore does not need to search from top to bottom, and a write-induced compaction operation ignores outdated records. The experiments show that for update intensive workloads, compared to widely-used key-value stores (e.g. RocksDB, Wisckey and PebblesDB), GHStore decreases read latency by 10%–50%, range query latency by 15%–60%, while increases write throughput by 4%–55%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baidu: Hugegraph. https://github.com/hugegraph/hugegraph (2019)

  2. Balmau, O., et al.: Triad: Creating synergies between memory, disk and log in log structured key-value stores. In: Proceedings of the 2017 USENIX Conference on Usenix Annual Technical Conference, pp. 363–375. USENIX ATC 2017, USENIX Association, USA (2017)

    Google Scholar 

  3. Bloom, B.H.: Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13(7), 422–426 (1970)

    Article  Google Scholar 

  4. Carbone, P., Ewen, S., Fóra, G., Haridi, S., Richter, S., Tzoumas, K.: State management in apache flink®: consistent stateful distributed stream processing. Proc. VLDB Endowment 10(12), 1718–1729 (2017)

    Article  Google Scholar 

  5. Chan, H.H., et al.: Hashkv: Enabling efficient updates in \(\{\)KV\(\}\) storage via hashing. In: 2018 \(\{\)USENIX\(\}\) Annual Technical Conference (\(\{\)USENIX\(\}\)\(\{\)ATC\(\}\) 18), pp. 1007–1019 (2018)

    Google Scholar 

  6. Comer, D.: Ubiquitous b-tree. ACM Comput. Surv. (CSUR) 11(2), 121–137 (1979)

    Article  MathSciNet  Google Scholar 

  7. Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing, pp. 143–154 (2010)

    Google Scholar 

  8. Dayan, N., Athanassoulis, M., Idreos, S.: Monkey: optimal navigable key-value store. In: Proceedings of the 2017 ACM International Conference on Management of Data, pp. 79–94 (2017)

    Google Scholar 

  9. DeCandia, G., et al.: Dynamo: amazon’s highly available key-value store. ACM SIGOPS Oper. Syst. Rev. 41(6), 205–220 (2007)

    Article  Google Scholar 

  10. Dong, S., Kryczka, A., Jin, Y., Stumm, M.: Rocksdb: evolution of development priorities in a key-value store serving large-scale applications. ACM Trans. Stor. (TOS) 17(4), 1–32 (2021)

    Article  Google Scholar 

  11. Facebook: Rocksdb. http://RocksDB.org (2017)

  12. Gade, A.N., Larsen, T.S., Nissen, S.B., Jensen, R.L.: Redis: a value-based decision support tool for renovation of building portfolios. Build. Environ. 142, 107–118 (2018)

    Article  Google Scholar 

  13. Ghemawat, S., Dean, J.: Leveldb. https://github.com/google/LevelDB (2011)

  14. Harter, T., et al.: Analysis of \(\{\)HDFS\(\}\) under hbase: a facebook messages case study. In: 12th \(\{\)USENIX\(\}\) Conference on File and Storage Technologies (\(\{\)FAST\(\}\) 14), pp. 199–212 (2014)

    Google Scholar 

  15. Huang, D., et al.: TIDB: a raft-based HTAP database. Proc. VLDB Endowment 13(12), 3072–3084 (2020)

    Article  Google Scholar 

  16. Jain, M.: Dgraph: synchronously replicated, transactional and distributed graph database. Birth (2005)

    Google Scholar 

  17. cockroach Labs: Cockroachdb. https://github.com/cockroachdb/cockroach (2017)

  18. Lai, C., Jiang, S., Yang, L., Lin, S., Cong, J.: Atlas: Baidu’s key-value storage system for cloud data. In: Symposium on Mass Storage Systems & Technologies, pp. 1–14 (2015)

    Google Scholar 

  19. Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. ACM SIGOPS Oper. Syst. Rev. 44(2), 35–40 (2010)

    Article  Google Scholar 

  20. Lin, Z., Kai, L., Cheng, Z., Wan, J.: Rangekv: An efficient key-value store based on hybrid dram-nvm-SSD storage structure. IEEE Access 8, 154518–154529 (2020)

    Article  Google Scholar 

  21. Lu, L., Pillai, T.S., Gopalakrishnan, H., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: Wisckey: separating keys from values in SSD-conscious storage. ACM Trans. Storage (TOS) 13(1), 1–28 (2017)

    Article  Google Scholar 

  22. MongoDB: Mongodb. https://github.com/mongodb/mongo (2017)

  23. O’Neil, P., Cheng, E., Gawlick, D., O’Neil, E.: The log-structured merge-tree (lsm-tree). Acta Inform. 33(4), 351–385 (1996)

    Google Scholar 

  24. Pan, F., Yue, Y., Xiong, J.: dcompaction: Delayed compaction for the lsm-tree. Int. J. Parallel Program. 45(6), 1310–1325 (2017)

    Article  Google Scholar 

  25. Popovitch, G.: parallel-hashmap. https://github.com/greg7mdp/parallel-hashmap (2020)

  26. Raju, P., Kadekodi, R., Chidambaram, V., Abraham, I.: Pebblesdb: building key-value stores using fragmented log-structured merge trees. In: Proceedings of the 26th Symposium on Operating Systems Principles, pp. 497–514 (2017)

    Google Scholar 

  27. Ren, K., Zheng, Q., Arulraj, J., Gibson, G.: Slimdb: a space-efficient key-value storage engine for semi-sorted data. Proc. VLDB Endowment 10(13), 2037–2048 (2017)

    Article  Google Scholar 

  28. Zhang, Q., Li, Y., Lee, P.P., Xu, Y., Cui, Q., Tang, L.: Unikv: toward high-performance and scalable kv storage in mixed workloads via unified indexing. In: 2020 IEEE 36th International Conference on Data Engineering (ICDE), pp. 313–324. IEEE (2020)

    Google Scholar 

  29. Zhang, W., Xu, Y., Li, Y., Zhang, Y., Li, D.: Flamedb: a key-value store with grouped level structure and heterogeneous bloom filter. IEEE Access 6, 24962–24972 (2018)

    Article  Google Scholar 

  30. Zhang, Z., et al.: Pipelined compaction for the LSM-tree. In: 2014 IEEE 28th International Parallel and Distributed Processing Symposium, pp. 777–786. IEEE (2014)

    Google Scholar 

Download references

Acknowledgements

We would like to thank the reviewers for their comments. This work was partially supported by BMKY2020B10.

Author information

Authors and Affiliations

  1. Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

    Jiaoyang Li, Yinliang Yue & Weiping Wang

  2. School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

    Jiaoyang Li, Yinliang Yue & Weiping Wang

Authors
  1. Jiaoyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinliang Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weiping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yinliang Yue .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Kanpur, Kanpur, India

    Arnab Bhattacharya

  2. National University of Singapore, Singapore, Singapore

    Janice Lee Mong Li

  3. University of California, Santa Barbara, Santa Barbara, CA, USA

    Divyakant Agrawal

  4. IIIT Hyderabad, Hyderabad, India

    P. Krishna Reddy

  5. Indraprastha Institute of Information Technology Delhi, New Delhi, India

    Mukesh Mohania

  6. Ashoka University, Sonepat, Haryana, India

    Anirban Mondal

  7. Indraprastha Institute of Information Technology Delhi, New Delhi, India

    Vikram Goyal

  8. University of Aizu, Aizu, Japan

    Rage Uday Kiran

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, J., Yue, Y., Wang, W. (2022). GHStore: A High Performance Global Hash Based Key-Value Store. In: Bhattacharya, A., et al. Database Systems for Advanced Applications. DASFAA 2022. Lecture Notes in Computer Science, vol 13245. Springer, Cham. https://doi.org/10.1007/978-3-031-00123-9_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-00123-9_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-00122-2

  • Online ISBN: 978-3-031-00123-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation