Abstract
All-Sources BFS (AS-BFS) is the main building block in a variety of complex network metric algorithms, such as the average path length and the betweenness centrality. However, AS-BFS calculations involve as many full BFS traversals as the total number of vertices, rendering AS-BFS impractical on commodity systems for real-world graphs with millions of vertices and links. In this paper we present our experience with the acceleration of AS-BFS graph metrics on multi-core HPC clusters by outlining hybrid coarse-grain parallel algorithms for computing the average path-length, the diameter and the betweenness centrality of complex networks in a lock-free fashion. We report speedups of up to 171\(\times \) on a heterogeneous cluster of 12-core Intel Xeon and 32-core AMD Opteron multi-core nodes; as well as resource utilizations of up to 75%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The BFS frontier keeps the nodes of the recently visited BFS level, being the \(i^{\text {th}}\) frontier the set of nodes at (shortest) distance i from the source node.
- 2.
Given two sets \(X=x_{1},x_{2},x_{3}..\) and \(Y=y_{1},y_{2},y_{3}..\), \(|X|=|Y|\), a vector sum is defined here as \(X\boxplus Y=x_{1}+y_{1},x_{2}+y_{2},x_{3}+y_{3}\ldots \) .
- 3.
The \(O(n+m)\) graph data structure can be shared among all the process’s threads.
- 4.
It only requires the communication of \(O(\eta )\) integers to the master process, with \(\eta =7\) and \(\eta =9\) for the Opteron cluster and the Opteron-Xeon cluster, respectively.
- 5.
It requires the communication of \(O(\eta )\) vectors of size O(n) to the master process.
- 6.
According to the 3-standard deviation rule of thumb, \(\pm \mu _{T}-3\sigma _{T}\) accounts for 99.73% of the time measurements, assuming that T is normally distributed.
- 7.
Average of speedups of the four experimented graphs for a given algorithm.
- 8.
Average of efficiencies of the four experimented graphs for a given algorithm.
- 9.
It requires maintaining a BFS stack, a queue and a predecessor list.
References
Bader, D.A., Kintali, S., Madduri, K., Mihail, M.: Approximating betweenness centrality. In: Bonato, A., Chung, F.R.K. (eds.) WAW 2007. LNCS, vol. 4863, pp. 124–137. Springer, Heidelberg (2007). doi:10.1007/978-3-540-77004-6_10
Bader, D.A., Madduri, K.: Parallel algorithms for evaluating centrality indices in real-world networks. In: Proceedings of 2006 International Conference on Parallel Processing, August 2006
Bader, D.A., Madduri, K.: A graph-theoretic analysis of the human protein-interaction network using multicore parallel algorithms. Parallel Comput. 34(11), 627–639 (2008)
Brandes, U.: A faster algorithm for betweenness centrality. J. Math. Sociol. 25(2), 163–177 (2001)
Committee on Network Science for Future Army Applications: Network Science. National Academies Press, Washington, DC (2005)
Costa, L.F., Rodrigues, F.A., Travieso, G., Boas, P.R.V.: Characterization of complex networks: a survey of measurements. Adv. Phys. 56(1), 167–242 (2007)
Edmonds, N., Hoefler, T., Lumsdaine, A.: A space-efficient parallel algorithm for computing betweenness centrality in distributed memory. In: Proceedings of 2010 International Conference on High Performance Computing, December 2010
Madduri, K., Ediger, D., Jiang, K., Bader, D.A., Chavarría-Miranda, D.: A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets. In: Proceedings of 2009 IEEE International Parallel and Distributed Processing Symposium, May 2009
Tan, G., Tu, D., Sun, N.: A parallel algorithm for computing betweenness centrality. In: Proceedings of 2009 International Conference on Parallel Processing, September 2009
Tu, D., Tan, G.: Characterizing betweenness centrality algorithm on multi-core architectures. In: Proceedings of 2009 IEEE International Symposium on Parallel and Distributed Processing with Applications, August 2009
Watts, D.J., Strogatz, S.H.: Collective dynamics of small-world networks. Nature 393(6684), 440–442 (1998)
Acknowledgments
The authors acknowledge to the General Coordination of Information and Communications Technologies (CGSTIC) at Cinvestav for providing HPC resources on the Hybrid Cluster Supercomputer “Xiuhcoatl”, that have contributed to the research results reported within this document.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Garcia-Robledo, A., Diaz-Perez, A., Morales-Luna, G. (2017). Accelerating All-Sources BFS Metrics on Multi-core Clusters for Large-Scale Complex Network Analysis. In: Barrios Hernández, C., Gitler, I., Klapp, J. (eds) High Performance Computing. CARLA 2016. Communications in Computer and Information Science, vol 697. Springer, Cham. https://doi.org/10.1007/978-3-319-57972-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-57972-6_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-57971-9
Online ISBN: 978-3-319-57972-6
eBook Packages: Computer ScienceComputer Science (R0)