Skip to main content
SpringerLink
Log in

NPGraph: An Efficient Graph Computing Model in NUMA-Based Persistent Memory Systems

  • Conference paper
  • First Online:
Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom 2023)

  • 142 Accesses

Abstract

The massive volume and the inherent imbalance of graphs are inevitable challenges for efficient graph computing, primarily due to the limited capacity of main memory (DRAM). Fortunately, a promising solution has emerged in the form of hybrid memory systems (HMS) which combine DRAM and persistent memory (PMEM) to enable data-centric graph computing. However, directly transitioning existing DRAM-based models to HMS can lead to inefficiency issues, especially when crossing Non-Uniform Memory Access (NUMA) nodes. In this paper, we present NPGraph, a novel approach that fully exploits the advantages of HMS for in-memory graph computing models. The main contributions of NPGraph lie in three aspects. Firstly, a dual-block graph representation strategy is devised to accelerate the process of subgraph construction. By utilizing data layering, it fully utilizes the storage architecture of HMS and optimizes the data access process. Secondly, an adaptive push-pull update strategy is proposed to optimize the message-updating process. With data-driven algorithms, it dynamically migrates subgraphs which are used in future iterations. Thirdly, the effectiveness of NPGraph is evaluated on five public graph data sets. Our model can improve the temporal locality and the spatial locality of graph computing concurrently. Extensive evaluation results show that NPGraph outperforms state-of-the-art graph computing models by 21.67%–32.03%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kyrola, A., Blelloch, G., Guestrin, C.: GraphChi: large-scale graph computation on just a PC. In: OSDI’12, pp. 31–46 (2012)

    Google Scholar 

  2. Sun, P., Wen, Y., Ta, D., Xiao, X.: GraphMP: I/O-efficient big graph analytics on a single commodity machine. IEEE Trans. Big Data, 2908384 (2019, to be published). https://doi.org/10.1109/TBDATA

  3. Zhang, Y., Yang, M., Baghdadi, R., et al.: Graphit: a high-performance graph DSL, In: Proceedings of the ACM on Programming Languages, vol. 2, no. OOPSLA, p. 121 (2018)

    Google Scholar 

  4. Malewicz, G., et al.: Pregel: a system for large-scale graph computing. In: SIGMOD’10, pp. 135–146 (2010)

    Google Scholar 

  5. Low, Y., et al.: Distributed GraphLab: a framework for machine learning in the cloud. Proc. VLDB Endow. 5(8) (2012)

    Google Scholar 

  6. Gonzalez, J.E., Xin, R.S., et al.: GraphX: graph computing in a distributed dataflow system. In: OSDI’14, pp. 599–613 (2014)

    Google Scholar 

  7. Shun, J., Blelloch, G.E.: Ligra: a lightweight graph computing system for shared memory. ACM SIGPLAN Not. 48(8), 135–146 (2013). ACM

    Google Scholar 

  8. Zhou, S.: Gemini: graph estimation with matrix variate normal instances. Ann. Stat. 42(2), 532–562 (2014)

    Google Scholar 

  9. Kumar, P., Huang, H.H.: Graphone: a data store for real-time analytics on evolving graphs. ACM Trans. Storage (TOS) 15(4), 1–40 (2020)

    Google Scholar 

  10. Gonzalez, J.E., et al.: PowerGraph: distributed graph-parallel computation on natural graphs. In: Usenix Conference on Operating Systems Design Implementation USENIX Association (2012)

    Google Scholar 

  11. https://www.intel.com/content/www/us/en/architecture-and-technology/optane-dc-persistent-memory.html

  12. Wang, R., et al.: XPGraph: XPline-friendly persistent memory graph stores for large-scale evolving graphs. In: 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE (2022)

    Google Scholar 

  13. Liu, W., Liu, H., Liao, X., Jin, H., Zhang, Y.: Straggler-aware parallel graph processing in hybrid memory systems. In: IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid). Melbourne, Australia, 2021, pp. 217–226 (2021). https://doi.org/10.1109/CCGrid51090.2021.00031

  14. Zhang, K., Chen, R., Chen, H.: Numa-aware graph-structured analytics. ACM SIGPLAN Not. 50(8), 183–193 (2015)

    Article  Google Scholar 

  15. Tian, Y., Balmin, A., Corsten, S.A., Tatikonda, S., McPherson, J.: From think like a vertex to think like a graph. Proc. VLDB Endow. 7(3), 193–204 (2013)

    Google Scholar 

  16. Huang, T., et al.: HyVE: hybrid vertex-edge memory hierarchy for energy-efficient graph processing. In: 2018 Design, Automation Test in Europe Conference Exhibition (DATE). IEEE (2018)

    Google Scholar 

  17. Gill, G., Dathathri, R., Hoang, L., et al.: Single machine graph analytics on massive datasets using Intel Optane DC persistent memory. Proc. VLDB Endow. 13(8), 1304–1318 (2020)

    Article  Google Scholar 

  18. Liu, H., Liu, R., Liao, X., Jin, H., He, B., Zhang, Y.: Object-level memory allocation and migration in hybrid memory systems. IEEE Trans. Comput. 69(9), 1401–1413 (2020). https://doi.org/10.1109/TC.2020.2973134

  19. Vora, K.: Lumos: dependency-driven disk-based graph processing. In: USENIX ATC, pp. 429–442 (2019)

    Google Scholar 

  20. Dang, Z., et al.: Nvalloc: rethinking heap metadata management in persistent memory allocators. In: ACM ASPLOS, pp. 115–127 (2022)

    Google Scholar 

  21. Wang, Q., Lu, Y., Li, J., Shu, J.: Nap: a black-box approach to NUMA-aware persistent memory indexes. In: USENIX OSDI, pp. 93–111 (2021)

    Google Scholar 

  22. http://snap.stanford.edu/data/ego-Facebook.html

  23. Backstrom, L., Huttenlocher, D., Kleinberg, J., Lan, X.: Group formation in large social networks: membership, growth, and evolution. In: KDD’06, pp. 44–54 (2006)

    Google Scholar 

  24. Kwak, H., Lee, C., Park, H., Moon, S.: What is Twitter, a social network or a news media. In: WWW’10, pp. 591–600 (2010)

    Google Scholar 

  25. Boldi, P., Vigna, S.: The webgraph system I: compression techniques. In: WWW’04, pp. 595–602 (2004)

    Google Scholar 

  26. http://developer.yahoo.com/blogs/616566076523839488/

  27. Li, B., et al.: D2Graph: an efficient and unified out-of-core graph computing model. In: 2021 IEEE ISPA, pp. 193–201 (2021). https://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00038

  28. Yu, J., et al.: DFOGraph: An I/O and Communication-Efficient System for Distributed Fully-out-of-Core Graph Processing (2021)

    Google Scholar 

  29. Liu, W., Liu, H., Liao, X., Jin, H., Zhang, Y: Straggler-aware parallel graph processing in hybrid memory systems. In: IEEE/ACM 21st International Symposium on Cluster. Cloud and Internet Computing (CCGrid) 2021, pp. 217–226 (2021). https://doi.org/10.1109/CCGrid51090.2021.00031

  30. Li, B., et al.: EPGraph: an efficient graph computing model in persistent memory system. In: 2022 IEEE ISPA, pp. 9–17 (2022). https://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom57177.2022.00009

Download references

Author information

Authors and Affiliations

  1. Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100085, China

    Baoke Li, Cong Cao, Fangfang Yuan, Yuling Yang & Yanbing Liu

  2. School of Cyber Security, University of Chinese Academy of Sciences, Beijing, 100085, China

    Baoke Li & Yuling Yang

  3. The 6th Research Institute of China Electronic Corporations, Beijing, China

    Majing Su

  4. Shandong Institutes of Industrial Technology, Jinan, China

    Jianhui Fu

Authors
  1. Baoke Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fangfang Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuling Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Majing Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yanbing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jianhui Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanbing Liu .

Editor information

Editors and Affiliations

  1. Shanghai University, Shanghai, China

    Honghao Gao

  2. Xi’an Jiaotong-Liverpool, Suzhou, China

    Xinheng Wang

  3. University of Peloponnese, Patra, Greece

    Nikolaos Voros

Rights and permissions

Reprints and permissions

Copyright information

© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, B. et al. (2024). NPGraph: An Efficient Graph Computing Model in NUMA-Based Persistent Memory Systems. In: Gao, H., Wang, X., Voros, N. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 562. Springer, Cham. https://doi.org/10.1007/978-3-031-54528-3_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-54528-3_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-54527-6

  • Online ISBN: 978-3-031-54528-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation