skip to main content
10.1145/3183713.3196924acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

Speeding Up Set Intersections in Graph Algorithms using SIMD Instructions

Published: 27 May 2018 Publication History

Abstract

In this paper, we focus on accelerating a widely employed computing pattern --- set intersection, to boost a group of graph algorithms. Graph's adjacency-lists can be naturally considered as node sets, thus set intersection is a primitive operation in many graph algorithms. We propose QFilter, a set intersection algorithm using SIMD instructions. QFilter adopts a merge-based framework and compares two blocks of elements iteratively by SIMD instructions. The key insight for our improvement is that we quickly filter out most of unnecessary comparisons in one byte-checking step. We also present a binary representation called BSR that encodes sets in a compact layout. By combining QFilter and BSR, we achieve data-parallelism in two levels --- inter-chunk and intra-chunk parallelism. Moreover, we find that node ordering impacts the performance of intersection by affecting the compactness of BSR. We formulate the graph reordering problem as an optimization of the compactness of BSR, and prove its strong NP-completeness. Thus we propose an approximate algorithm that can find a better ordering to enhance the intra-chunk parallelism. We conduct extensive experiments to confirm that our approach can improve the performance of set intersection in graph algorithms significantly.

References

[1]
Christopher R Aberger, Andrew Lamb, Susan Tu, Andres Nötzli, Kunle Olukotun, and Christopher Ré. 2017. Emptyheaded: A relational engine for graph processing. TODS, Vol. 42, 4 (2017), 20.
[2]
Konstantin Andreev and Harald Racke. 2006. Balanced graph partitioning. TCS, Vol. 39, 6 (2006), 929--939.
[3]
Naiyong Ao, Fan Zhang, Di Wu, Douglas S Stones, Gang Wang, Xiaoguang Liu, Jing Liu, and Sheng Lin. 2011. Efficient parallel lists intersection and index compression algorithms using graphics processing units. PVLDB, Vol. 4, 8 (2011), 470--481.
[4]
Ricardo Baeza-Yates. 2004. A fast set intersection algorithm for sorted sequences CPM. Springer, 400--408.
[5]
Jérémy Barbay and Claire Kenyon. 2002. Adaptive intersection and t-threshold problems. In SODA. Society for Industrial and Applied Mathematics, 390--399.
[6]
Daniel K Blandford, Guy E Blelloch, and Ian A Kash . 2003. Compact representations of separable graphs. In SODA. Society for Industrial and Applied Mathematics, 679--688.
[7]
Guy E Blelloch and Margaret Reid-Miller. 1998Fast set operations using treaps. In ACM SPAA. ACM, 16--26.
[8]
Coen Bron and Joep Kerbosch. 1973. Finding all cliques of an undirected graph (algorithm 457). Commun. ACM, Vol. 16, 9 (1973), 575--577.
[9]
Yangjun Chen and Weixin Shen. 2016. An efficient method to evaluate intersections on big data sets. TCS Vol. 647 (2016), 1--21.
[10]
Flavio Chierichetti, Ravi Kumar, Silvio Lattanzi, Michael Mitzenmacher, Alessandro Panconesi, and Prabhakar Raghavan. 2009. On compressing social networks. In SIGKDD. ACM, 219--228.
[11]
Shumo Chu and James Cheng. 2011. Triangle listing in massive networks and its applications SIGKDD. ACM, 672--680.
[12]
Luigi Pietro Cordella, Pasquale Foggia, Carlo Sansone, and Mario Vento. 2004. A (sub) graph isomorphism algorithm for matching large graphs. TPAMI, Vol. 26, 10 (2004), 1367--1372.
[13]
Wanyun Cui, Yanghua Xiao, Haixun Wang, Yiqi Lu, and Wei Wang. 2013. Online search of overlapping communities. In SIGMOD. ACM, 277--288.
[14]
Erik D Demaine, Alejandro López-Ortiz, and J Ian Munro. 2000. Adaptive set intersections, unions, and differences SODA. Citeseer.
[15]
Laxman Dhulipala, Igor Kabiljo, Brian Karrer, Giuseppe Ottaviano, Sergey Pupyrev, and Alon Shalita. 2016. Compressing graphs and indexes with recursive graph bisection SIGKDD. ACM, 1535--1544.
[16]
Bolin Ding and Arnd Christian König. 2011. Fast set intersection in memory. PVLDB, Vol. 4, 4 (2011), 255--266.
[17]
David Eppstein, Maarten Löffler, and Darren Strash. 2010. Listing all maximal cliques in sparse graphs in near-optimal time ISAAC. Springer, 403--414.
[18]
Ziqiang Feng, Eric Lo, Ben Kao, and Wenjian Xu. 2015. Byteslice: Pushing the envelop of main memory data processing with a new storage layout SIGMOD. ACM, 31--46.
[19]
Marta Fort, J Antoni Sellarès, and Nacho Valladares. 2017. Intersecting two families of sets on the GPU. JPDC Vol. 104 (2017), 167--178.
[20]
Michael R Gary and David S Johnson. 1979. Computers and Intractability: A Guide to the Theory of NP-completeness. (1979).
[21]
Wook-Shin Han, Jinsoo Lee, and Jeong-Hoon Lee. 2013. Turbo iso: towards ultrafast and robust subgraph isomorphism search in large graph databases SIGMOD. ACM, 337--348.
[22]
Xin Huang, Hong Cheng, Lu Qin, Wentao Tian, and Jeffrey Xu Yu. 2014. Querying k-truss community in large and dynamic graphs SIGMOD. ACM, 1311--1322.
[23]
Hiroshi Inoue, Moriyoshi Ohara, and Kenjiro Taura. 2014. Faster set intersection with simd instructions by reducing branch mispredictions. PVLDB, Vol. 8, 3 (2014), 293--304.
[24]
George Karypis and Vipin Kumar. 1998. Multilevelk-way partitioning scheme for irregular graphs. J. Parallel Distrib. Comput. Vol. 48, 1 (1998), 96--129.
[25]
Ilya Katsov. 2012. Fast intersection of sorted lists using SSE instructions. (2012). https://highlyscalable.wordpress.com/2012/06/05/fast-intersection-sorted-lists-sse/
[26]
Jérôme Kunegis. 2013. Konect: the koblenz network collection. In WWW. ACM, 1343--1350.
[27]
Daniel Lemire, Leonid Boytsov, and Nathan Kurz. 2016. SIMD compression and the intersection of sorted integers. SPE, Vol. 46, 6 (2016), 723--749.
[28]
Daniel Lemire, Owen Kaser, Nathan Kurz, Luca Deri, Chris O'Hara, Franccois Saint-Jacques, and Gregory Ssi-Yan-Kai. 2017. Roaring bitmaps: Implementation of an optimized software library. SPE (2017).
[29]
Jure Leskovec and Andrej Krevl. 2014. SNAP Datasets: Stanford Large Network Dataset Collection. http://snap.stanford.edu/data. (2014).
[30]
Yongsub Lim, U Kang, and Christos Faloutsos. 2014. Slashburn: Graph compression and mining beyond caveman communities. TKDE, Vol. 26, 12 (2014), 3077--3089.
[31]
Gergely Palla, Imre Derényi, Illés Farkas, and Tamás Vicsek. 2005. Uncovering the overlapping community structure of complex networks in nature and society. Nature, Vol. 435, 7043 (2005), 814--818.
[32]
Benjamin Schlegel, Thomas Willhalm, and Wolfgang Lehner. 2011. Fast sorted-Set intersection using SIMD instructions ADMS@VLDB. 1--8.
[33]
Julian Shun. 2017. Shared-memory parallelism can be simple, fast, and scalable. Morgan &Claypool.
[34]
Julian Shun and Kanat Tangwongsan. 2015. Multicore triangle computations without tuning. In ICDE. IEEE, 149--160.
[35]
Shirish Tatikonda, Flavio Junqueira, B Barla Cambazoglu, and Vassilis Plachouras . 2009. On efficient posting list intersection with multicore processors SIGIR. ACM, 738--739.
[36]
Dimitris Tsirogiannis, Sudipto Guha, and Nick Koudas. 2009. Improving the performance of list intersection. PVLDB, Vol. 2, 1 (2009), 838--849.
[37]
Charalampos Tsourakakis, Francesco Bonchi, Aristides Gionis, Francesco Gullo, and Maria Tsiarli. 2013. Denser than the densest subgraph: extracting optimal quasi-cliques with quality guarantees SIGKDD. ACM, 104--112.
[38]
Jianguo Wang, Chunbin Lin, Yannis Papakonstantinou, and Steven Swanson. 2017. An Experimental Study of Bitmap Compression vs. Inverted List Compression SIGMOD. ACM, 993--1008.
[39]
Nan Wang, Jingbo Zhang, Kian-Lee Tan, and Anthony KH Tung. 2010. On triangulation-based dense neighborhood graph discovery. PVLDB, Vol. 4, 2 (2010), 58--68.
[40]
Hao Wei, Jeffrey Xu Yu, Can Lu, and Xuemin Lin. 2016. Speedup graph processing by graph ordering. In SIGMOD. ACM, 1813--1828.
[41]
Thomas Willhalm, Nicolae Popovici, Yazan Boshmaf, Hasso Plattner, Alexander Zeier, and Jan Schaffner. 2009. SIMD-scan: ultra fast in-memory table scan using on-chip vector processing units. PVLDB, Vol. 2, 1 (2009), 385--394.
[42]
Wenjian Xu, Ziqiang Feng, and Eric Lo. 2016. Fast multi-column sorting in main-memory column-stores SIGMOD. ACM, 1263--1278.
[43]
Jingren Zhou and Kenneth A Ross. 2002. Implementing database operations using SIMD instructions SIGMOD. ACM, 145--156.

Cited By

View all
  • (2024)HERO: A Hierarchical Set Partitioning and Join Framework for Speeding up the Set Intersection Over GraphsProceedings of the ACM on Management of Data10.1145/36392842:1(1-25)Online publication date: 26-Mar-2024
  • (2024)Fast Subgraph Matching by Dynamic Graph EditingIEEE Transactions on Services Computing10.1109/TSC.2023.333384017:5(2432-2443)Online publication date: Sep-2024
  • (2024)BigSet: An Efficient Set Intersection ApproachIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.343259536:12(7677-7691)Online publication date: Dec-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGMOD '18: Proceedings of the 2018 International Conference on Management of Data
May 2018
1874 pages
ISBN:9781450347037
DOI:10.1145/3183713
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. graph ordering
  2. graph processing
  3. set intersection
  4. simd

Qualifiers

  • Research-article

Funding Sources

  • the Research Grant Council of Hong Kong SAR China
  • National Natural Science Foundation of China
  • the National Key Research and Development Program of China

Conference

SIGMOD/PODS '18
Sponsor:

Acceptance Rates

SIGMOD '18 Paper Acceptance Rate 90 of 461 submissions, 20%;
Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)147
  • Downloads (Last 6 weeks)9
Reflects downloads up to 30 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)HERO: A Hierarchical Set Partitioning and Join Framework for Speeding up the Set Intersection Over GraphsProceedings of the ACM on Management of Data10.1145/36392842:1(1-25)Online publication date: 26-Mar-2024
  • (2024)Fast Subgraph Matching by Dynamic Graph EditingIEEE Transactions on Services Computing10.1109/TSC.2023.333384017:5(2432-2443)Online publication date: Sep-2024
  • (2024)BigSet: An Efficient Set Intersection ApproachIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.343259536:12(7677-7691)Online publication date: Dec-2024
  • (2024)CRISP: Triangle Counting Acceleration via Content Addressable Memory-Integrated 3D-Stacked Memory2024 IEEE International Test Conference in Asia (ITC-Asia)10.1109/ITC-Asia62534.2024.10661308(1-6)Online publication date: 18-Aug-2024
  • (2024)IVE: Accelerating Enumeration-Based Subgraph Matching via Exploring Isolated Vertices2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00321(4208-4221)Online publication date: 13-May-2024
  • (2024)GPU-Accelerated Batch-Dynamic Subgraph Matching2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00248(3204-3216)Online publication date: 13-May-2024
  • (2024) G 2 -AIMD: A Memory-Efficient Subgraph-Centric Framework for Efficient Subgraph Finding on GPUs 2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00245(3164-3177)Online publication date: 13-May-2024
  • (2024)Accelerating Maximal Bicliques Enumeration with GPU on large scale networkFuture Generation Computer Systems10.1016/j.future.2024.07.021Online publication date: Jul-2024
  • (2024)$$(\alpha ,\beta )$$-Butterfly Computation on Bipartite GraphsArabian Journal for Science and Engineering10.1007/s13369-024-09632-zOnline publication date: 21-Oct-2024
  • (2024)A graph pattern mining framework for large graphs on GPUThe VLDB Journal10.1007/s00778-024-00883-834:1Online publication date: 5-Dec-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media