Skip to main content
SpringerLink
Log in
Book cover

International Conference on Web Information Systems Engineering

WISE 2021: Web Information Systems Engineering – WISE 2021 pp 501–509Cite as

GPUGraphX: A GPU-Aided Distributed Graph Processing System

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13081))

Abstract

There are two major challenges for large-scale graph analytic processing, computational intensiveness caused by complex graph primitives and distributed data management caused by data of massive scales. Existing works on graph data management with CPU-based distributed systems or GPU-based single-node systems only partially solve the problem. Hence, it is desired to have a general graph processing system for both scaling out and scaling up. In this paper, we demonstrate GPUGraphX, a GPU-aided distributed graph processing system which utilizes computation capacities of GPUs for efficiency while taking the advantages of distributed systems for scalability. Results on representative graph algorithms on real datasets evaluate our proposals.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Code Repository: https://github.com/Kamosphere/spark-GPUGraphX.

  2. 2.

    The four parts correspond to the four stages of a GraphX iteration during the execution, aggregateMessageswithActiveSet@GraphImpl.scala, shipVertexAttributes@VertexRDDImpl.scala, shipVertexIds@VertexRDDImpl.scala, and count@VertexRDDImpl.scala.

References

  1. Batarfi, O., El Shawi, R., et al.: Large scale graph processing systems: survey and an experimental evaluation. Clust. Comput. 18(3), 1189–1213 (2015)

    Google Scholar 

  2. Bauer, M., Treichler, S., et al.: Legion: expressing locality and independence with logical regions. In: SC, p. 66 (2012)

    Google Scholar 

  3. Gill, G., Dathathri, R., et al.: Abelian: a compiler for graph analytics on distributed, heterogeneous platforms. In: Euro-Par, pp. 249–264 (2018)

    Google Scholar 

  4. Gonzalez, J.E., Xin, R.S., et al.: Graphx: graph processing in a distributed dataflow framework. In: OSDI, pp. 599–613 (2014)

    Google Scholar 

  5. Gonzalez, J.E., Low, Y., et al.: Powergraph: distributed graph-parallel computation on natural graphs. In: OSDI, pp. 17–30 (2012)

    Google Scholar 

  6. Gregor, D., Lumsdaine, A.: The parallel BGL: a generic library for distributed graph computations. In: POOSC (2005)

    Google Scholar 

  7. Han, M., Daudjee, K., et al.: An experimental comparison of pregel-like graph processing systems. PVLDB 7(12), 1047–1058 (2014)

    Google Scholar 

  8. Jia, Z., Kwon, Y., et al.: A distributed multi-GPU system for fast graph processing. VLDB 11(3), 297–310 (2017)

    Google Scholar 

  9. Leskovec, J., Krevl, A.: SNAP datasets: Stanford large network dataset collection (2014). http://snap.stanford.edu/data

  10. Liu, Y., Xie, X.: Xy-sketch: on sketching data streams at web scale. In: WWW, pp. 1169–1180 (2021)

    Google Scholar 

  11. Lu, Y., Cheng, J., et al.: Large-scale distributed graph computing systems: an experimental evaluation. PVLDB 8(3), 281–292 (2014)

    Google Scholar 

  12. Luo, J., Cao, X., Xie, X., Qu, Q., Xu, Z., Jensen, C.S.: Efficient attribute-constrained co-located community search. In: ICDE, pp. 1201–1212 (2020)

    Google Scholar 

  13. Malewicz, G., Austern, M.H., et al.: Pregel: a system for large-scale graph processing. In: SIGMOD, pp. 135–146 (2010)

    Google Scholar 

  14. Mislove, A., Marcon, M., et al.: Measurement and analysis of online social networks. In: IMC (2007)

    Google Scholar 

  15. Rossi, R.A., Ahmed, N.K.: The network data repository with interactive graph analytics and visualization. In: AAAI (2015). http://networkrepository.com

  16. Saleem, M.A., Kumar, R., Calders, T., Xie, X., Pedersen, T.B.: Location influence in location-based social networks. In: WSDM, pp. 621–630 (2017)

    Google Scholar 

  17. Shun, J., Blelloch, G.E.: Ligra: a lightweight graph processing framework for shared memory. In: PPoPP, pp. 135–146 (2013)

    Google Scholar 

  18. Wang, Y., Davidson, A.A., et al.: Gunrock: a high-performance graph processing library on the GPU. In: PPoPP, pp. 11:1–11:12 (2016)

    Google Scholar 

  19. Watkins, N.: Programmable storage. Ph.D. thesis, University of California, Santa Cruz, USA (2018)

    Google Scholar 

  20. Wu, F., Xie, X., Shi, J.: Top-k closest pair queries over spatial knowledge graph. In: DASFAA, pp. 625–640 (2021)

    Google Scholar 

  21. Xie, X., Mei, B., Chen, J., Du, X., Jensen, C.S.: Elite: an elastic infrastructure for big spatiotemporal trajectories. VLDB J. 25(4), 473–493 (2016)

    Article  Google Scholar 

  22. Xie, X., Yiu, M.L., Cheng, R., Lu, H.: Scalable evaluation of trajectory queries over imprecise location data. TKDE 26(8), 2029–2044 (2014)

    Google Scholar 

  23. Zhong, J., He, B.: Medusa: a parallel graph processing system on graphics processors. SIGMOD Rec. 43(2), 35–40 (2014)

    Article  Google Scholar 

Download references

Acknowledgments

This work is partially supported by NSFC (No. 61772492, 62072428) and the CAS Pioneer Hundred Talents Program. The numerical calculations in this paper have been done on the supercomputing system in the Supercomputing Center of University of Science and Technology of China.

Author information

Authors and Affiliations

  1. University of Science and Technology of China, Hefei, China

    Qi Li, Kai Zou, Deyu Kong, Huhao Guan & Xike Xie

Authors
  1. Qi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deyu Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huhao Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xike Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xike Xie .

Editor information

Editors and Affiliations

  1. School of Computer Science and Engineering, The University of New South Wales, Sydney, NSW, Australia

    Wenjie Zhang

  2. Peking University, Beijing, China

    Lei Zou

  3. Zayed University, Dubai, United Arab Emirates

    Zakaria Maamar

  4. Swinburne University of Technology, Melbourne, VIC, Australia

    Lu Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, Q., Zou, K., Kong, D., Guan, H., Xie, X. (2021). GPUGraphX: A GPU-Aided Distributed Graph Processing System. In: Zhang, W., Zou, L., Maamar, Z., Chen, L. (eds) Web Information Systems Engineering – WISE 2021. WISE 2021. Lecture Notes in Computer Science(), vol 13081. Springer, Cham. https://doi.org/10.1007/978-3-030-91560-5_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-91560-5_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-91559-9

  • Online ISBN: 978-3-030-91560-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation