Abstract
Deploying large scale knowledge graphs on distributed systems has become an industry trend for their high scalability and availability. There are some distributed graph databases that prefer to adopt the NoSQL data models like the key-value store as their storage engines for its scalability and practicability. Therefore, an upper-level graph query language (GQL) statement will be translated into a group of the native and hybrid kinds of key-value (KV) operations. To accelerate the KV operations generated form upper-level knowledge graph queries, we propose a high performance knowledge graph system with a non-volatile memory based queries booster (KGB). KGB mainly contains a neighbors queries auxiliary index for reducing KVs searching cost, a fast Raft algorithm for efficient KVs operations, and a KV tuning mechanism to acquire extra performance promotion for knowledge graph application scenarios. Experiments show that KGB can effectively reduce the latency and achieve higher performance promotion for knowledge graph system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sbadger: A fast key-value store written natively in go (2020). https://github.com/dgraph-io/badger
Balmau, O., Dinu, F., Zwaenepoel, W., Gupta, K., Chandhiramoorthi, R., Didona, D.: SILK: preventing latency spikes in log-structured merge key-value stores. In: 2019 USENIX Annual Technical Conference, pp. 753–766 (2019)
Buragohain, C., et al.: A1: a distributed in-memory graph database. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, pp. 329–344 (2020)
Cao, Z., Dong, S., Vemuri, S., Du, D.H.: Characterizing, modeling, and benchmarking rocksdb key-value workloads at facebook. In: 18th \(\{\)USENIX\(\}\) Conference on File and Storage Technologies (\(\{\)FAST\(\}\) 2020), pp. 209–223 (2020)
Chai, Y., Chai, Y., Wang, X., Wei, H., Bao, N., Liang, Y.: LDC: a lower-level driven compaction method to optimize SSD-oriented key-value stores. In: 2019 IEEE 35th International Conference on Data Engineering (ICDE), pp. 722–733. IEEE (2019)
Chai, Y., Chai, Y., Wang, X., Wei, H., Wang, Y.: Adaptive lower-level driven compaction to optimize LSM-tree key-value stores. IEEE Trans. Knowl. Data Eng. 34(6), 2595–2609 (2020)
Dageville, B., et al.: The snowflake elastic data warehouse. In: Proceedings of the 2016 International Conference on Management of Data, pp. 215–226 (2016)
Dayan, N., Athanassoulis, M., Idreos, S.: Monkey: optimal navigable key-value store. In: SIGMOD, pp. 79–94. ACM (2017)
Dong, S., Callaghan, M., Galanis, L., Borthakur, D., Savor, T., Strum, M.: Optimizing space amplification in rocksdb. In: CIDR (2017)
Gao, S., et al.: Formal verification of consensus in the taurus distributed database. In: Huisman, M., Păsăreanu, C., Zhan, N. (eds.) FM 2021. LNCS, vol. 13047, pp. 741–751. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90870-6_42
Gupta, P., Mhedhbi, A., Salihoglu, S.: Columnar storage and list-based processing for graph database management systems. Proc. VLDB Endow. 14(11), 2491–2504 (2021)
Jain, M.: Dgraph: synchronously replicated, transactional and distributed graph database. birth (2005)
Leveldb - a fast and lightweight key/value database library by google (2017). http://code.google.com/p/leveldb
Li, S., Chen, W., Liu, B., Liu, P., Wang, X., Li, Y.-F.: OntoSP: ontology-based semantic-aware partitioning on RDF graphs. In: Zhang, W., Zou, L., Maamar, Z., Chen, L. (eds.) WISE 2021. LNCS, vol. 13080, pp. 258–273. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90888-1_21
Matsunobu, Y., Dong, S., Lee, H.: Myrocks: LSM-tree database storage engine serving Facebook’s social graph. Proc. VLDB Endow. 13(12), 3217–3230 (2020)
Miller, J.J.: Graph database applications and concepts with neo4j. In: Proceedings of the Southern Association for Information Systems Conference, Atlanta, GA, USA, vol. 2324 (2013)
Under the hood: Building and open-sourcing rocksdb (2017). http://goo.gl/9xulVB
Ruta, N.J.: CuttleTree: adaptive tuning for optimized log-structured merge trees. Ph.D. thesis, Harvard University (2017)
Sarkar, S., Papon, T.I., Staratzis, D., Athanassoulis, M.: Lethe: a tunable delete-aware LSM engine. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, pp. 893–908 (2020)
Urbani, J., Jacobs, C.: Adaptive low-level storage of very large knowledge graphs. In: Proceedings of The Web Conference 2020, pp. 1761–1772 (2020)
Wang, Y., Chai, Y.: vRaft: accelerating the distributed consensus under virtualized environments. In: Jensen, C.S., Lim, E.-P., Yang, D.-N., Lee, W.-C., Tseng, V.S., Kalogeraki, V., Huang, J.-W., Shen, C.-Y. (eds.) DASFAA 2021. LNCS, vol. 12681, pp. 53–70. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73194-6_4
Wang, Y., Wang, Z., Chai, Y., Wang, X.: Rethink the linearizability constraints of raft for distributed key-value stores. In: 2021 IEEE 37th International Conference on Data Engineering (ICDE), pp. 1877–1882. IEEE (2021)
Wu, F., Yang, M.H., Zhang, B., Du, D.H.: AC-key: adaptive caching for LSM-based key-value stores. In: 2020 \(\{\)USENIX\(\}\) Annual Technical Conference (\(\{\)USENIX\(\}\)\(\{\)ATC\(\}\) 2020), pp. 603–615 (2020)
Acknowledgements
This work is supported by the National Key Research and Development Program of China (No. 2019YFE0198600), National Natural Science Foundation of China (No. 61972402, 61972275, and 61732014).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, Y., Li, W., Wang, Y., Chai, Y., Chai, Y. (2023). Accelerating Distributed Knowledge Graph System Based on Non-Volatile Memory. In: Yang, S., Islam, S. (eds) Web and Big Data. APWeb-WAIM 2022 International Workshops. APWeb-WAIM 2022. Communications in Computer and Information Science, vol 1784. Springer, Singapore. https://doi.org/10.1007/978-981-99-1354-1_3
Download citation
DOI: https://doi.org/10.1007/978-981-99-1354-1_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1353-4
Online ISBN: 978-981-99-1354-1
eBook Packages: Computer ScienceComputer Science (R0)