Abstract
With the exponential growth of graph structured data in recent years, parallel distributed techniques play an increasingly important role in processing large-scale graphs. Since strong connections exist between vertices in graph data, the high communication cost for transforming boundary data is unavoidable in the distributed techniques. How to partition a large graph into several partitions with low coupling and balanced scale becomes a critical problem. Most of research in the literature studies vertex partitioning methods, which leads us to reconsider an alternative approach for edge partitioning. In this paper, we propose a distributed algorithm for graph partition based on edge partitioning, named as VSEP. A novel vertex permutation method is used to partition the large graphs iteratively. Experimental results indicate that VSEP reduces the number of times vertices are cut by about \(10\,\%\sim 20\,\%\) comparing with a state-of-the-art algorithm while retains the scale balance.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Malewicz, G., Austern, M.H., Bik, A.J.C., et al.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pp. 135–146. ACM (2010)
Low, Y., Gonzalez, J.E., Kyrola, A., et al.: Graphlab: a new framework for parallel machine learning (2014). arXiv preprint arXiv:1408.2041
Gonzalez, J.E., Low, Y., Gu, H., et al.: PowerGraph: distributed graph-parallel computation on natural graphs. In: OSDI, vol. 12(1), p. 2 (2012)
Xin, R.S., Gonzalez, J.E., Franklin, M.J., et al.: Graphx: a resilient distributed graph system on spark. In: First International Workshop on Graph Data Management Experiences and Systems, p. 2. ACM (2013)
Garey, M.R., Johnson, D.S., Stockmeyer, L.: Some simplified NP-complete graph problems. Theoret. Comput. Sci. 1(3), 237–267 (1976)
Abou-Rjeili, A., Karypis, G.: Multilevel algorithms for partitioning power-law graphs. In: 2006 20th International Parallel and Distributed Processing Symposium, IPDPS 2006, p. 10. IEEE (2006)
Lang, K.: Finding good nearly balanced cuts in power law graphs (2004). Preprint
Leskovec, J., Lang, K.J., Dasgupta, A., et al.: Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. Internet Math. 6(1), 29–123 (2009)
Kim, M., Candan, K.S.: SBV-Cut: vertex-cut based graph partitioning using structural balance vertices. Data Knowl. Eng. 72, 285–303 (2012)
Guerrieri, A., Montresor, A.: Distributed edge partitioning for graph processing (2014). arXiv preprint arXiv:1403.6270
Rahimian, F., Payberah, A.H., Girdzijauskas, S., Haridi, S.: Distributed vertex-cut partitioning. In: Magoutis, K., Pietzuch, P. (eds.) DAIS 2014. LNCS, vol. 8460, pp. 186–200. Springer, Heidelberg (2014)
Albert, R., Jeong, H., Barabsi, A.L.: Error and attack tolerance of complex networks. Nature 406(6794), 378–382 (2000)
Rahimian, F., Payberah, A.H., Girdzijauskas, S., et al.: Ja-be-ja: a distributed algorithm for balanced graph partitioning. In: 2013 IEEE 7th International Conference on Self-Adaptive and Self-Organizing Systems (SASO), pp. 51–60. IEEE (2013)
Baños, R., Gil, C., Ortega, J., Montoya, F.G.: Multilevel heuristic algorithm for graph partitioning. In: Raidl, G.R., et al. (eds.) EvoIASP 2003, EvoWorkshops 2003, EvoSTIM 2003, EvoROB/EvoRobot 2003, EvoCOP 2003, EvoBIO 2003, and EvoMUSART 2003. LNCS, vol. 2611, pp. 143–153. Springer, Heidelberg (2003)
Bui, T.N., Moon, B.R.: Genetic algorithm and graph partitioning. IEEE Trans. Comput. 45(7), 841–855 (1996)
Hendrickson, B., Leland, R.: A multi-level algorithm for partitioning graphs (1995)
Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1998)
Karypis, G., Kumar, V.: Parallel multilevel series k-way partitioning scheme for irregular graphs. SIAM Rev. 41(2), 278–300 (1999)
Walshaw, C., Cross, M.: Mesh partitioning: a multilevel balancing and refinement algorithm. SIAM J. Sci. Comput. 22(1), 63–80 (2000)
Sanders, P., Schulz, C.: Engineering multilevel graph partitioning algorithms. In: Demetrescu, C., Halldórsson, M.M. (eds.) ESA 2011. LNCS, vol. 6942, pp. 469–480. Springer, Heidelberg (2011)
Soper, A.J., Walshaw, C., Cross, M.: A combined evolutionary search and multilevel optimisation approach to graph-partitioning. J. Global Optim. 29(2), 225–241 (2004)
Chardaire, P., Barake, M., McKeown, G.P.: A PROBE-based heuristic for graph partitioning. IEEE Trans. Comput. 56(12), 1707–1720 (2007)
Sanders, P., Schulz, C.: Distributed evolutionary graph partitioning. In: ALENEX, pp. 16–29 (2012)
Talbi, E.G.: Metaheuristics: From Design to Implementation. Wiley, New York (2009)
Carlini, E., Dazzi, P., Esposito, A., Lulli, A., Ricci, L.: Balanced graph partitioning with apache spark. In: Žilinskas, J., et al. (eds.) Euro-Par 2014, Part I. LNCS, vol. 8805, pp. 129–140. Springer, Heidelberg (2014)
Zaharia, M., Chowdhury, M., Franklin, M.J., et al.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, p. 10 (2010)
The graph partitioning archive. http://staffweb.cms.gre.ac.uk/~wc06/partition
Stanford large network dataset collection. http://snap.stanford.edu/data/index.html
Acknowledgment
This research was supported by the National Natural Science Foundation of China (No. 61202477 and No. 61272427); the Strategic Priority Research Program of the Chinese Academy of Sciences (No. XDA06031000).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Zhang, Y., Liu, Y., Yu, J., Liu, P., Guo, L. (2015). VSEP: A Distributed Algorithm for Graph Edge Partitioning. In: Wang, G., Zomaya, A., Martinez, G., Li, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2015. Lecture Notes in Computer Science(), vol 9532. Springer, Cham. https://doi.org/10.1007/978-3-319-27161-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-27161-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27160-6
Online ISBN: 978-3-319-27161-3
eBook Packages: Computer ScienceComputer Science (R0)