He Li
ABSTRACT
To improve the performance of large graph computing, graph partitioning has become a mandatory step in distributed graph computing frameworks. Some existing frameworks partition edges of an input graph in a streaming way. As the scale of real-world graphs grows dynamically, they need to limit the increasing communication cost and time cost in graph computing by reducing vertex replicas(each vertex can be replicated to multiple partitions). In this paper, we propose a real-time edge repartitioning algorithm for dynamic graph, which reduces the vertex replicas by reassigning edges near the new edge. We find that some edges are migrated just after being assigned, which leads to unnecessary migrations. To reduce migration cost, according to the replicas distribution of neighbors of two vertices connected by the new edge, we assign the new edge to the partition where it is most likely to be located after repartitioning. Our evaluation shows that it improves the performance of graph computing by only a small amount of migration.
- Florian Bourse, Marc Lelarge, and Milan Vojnovic. 2014. Balanced graph edge partition. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 1456--1465.Google Scholar
Digital Library
- Joseph E Gonzalez, Yucheng Low, Haijie Gu, Danny Bickson, and Carlos Guestrin. 2012. Powergraph: Distributed graph-parallel computation on natural graphs. In Presented as part of the 10th $$USENIX$$ Symposium on Operating Systems Design and Implementation ($$OSDI$$ 12). 17--30.Google Scholar
- Dinesh Kumar, Arun Raj, and Janakiram Dharanipragada. 2017. GraphSteal: Dynamic Re-Partitioning for Efficient Graph Processing in Heterogeneous Clusters. In 2017 IEEE 10th International Conference on Cloud Computing (CLOUD). IEEE, 439--446.Google Scholar
- Jure Leskovec and Andrej Krevl. 2014. Stanford Large Network Dataset Collection. http://snap.stanford.edu/data .Google Scholar
- Yucheng Low, Danny Bickson, Joseph Gonzalez, Carlos Guestrin, Aapo Kyrola, and Joseph M Hellerstein. 2012. Distributed GraphLab: a framework for machine learning and data mining in the cloud. Proceedings of the VLDB Endowment , Vol. 5, 8 (2012), 716--727.Google ScholarDigital Library
- Grzegorz Malewicz, Matthew H Austern, Aart JC Bik, James C Dehnert, Ilan Horn, Naty Leiser, and Grzegorz Czajkowski. 2010. Pregel: a system for large-scale graph processing. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data. ACM, 135--146.Google ScholarDigital Library
- Christian Mayer, Muhammad Adnan Tariq, Ruben Mayer, and Kurt Rothermel. 2018. Graph: Traffic-aware graph processing. IEEE Transactions on Parallel and Distributed Systems , Vol. 29, 6 (2018), 1289--1302.Google ScholarCross Ref
- Fabio Petroni, Leonardo Querzoni, Khuzaima Daudjee, Shahin Kamali, and Giorgio Iacoboni. 2015. Hdrf: Stream-based partitioning for power-law graphs. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 243--252.Google ScholarDigital Library
Index Terms
- Real-time Edge Repartitioning for Dynamic Graph
Recommendations
Collapsible subgraphs of a 4-edge-connected graph
AbstractJaeger in 1979 showed that every 4-edge-connected graph is supereulerian, graphs that have spanning eulerian subgraphs. Catlin in 1988 sharpened Jaeger’s result by showing that every 4-edge-connected graph is collapsible, graphs that ...
Graph edge colouring: Tashkinov trees and Goldberg's conjecture
For the chromatic index @g^'(G) of a (multi)graph G, there are two trivial lower bounds, namely the maximum degree @D(G) and the density W(G)=max"H"@__ __"G","|"V"("H")"|">="2@__ __|E(H)|/@__ __|V(H)|/2@__ __@__ __. A famous conjecture due to Goldberg [...
Computing a Minimum Cut in a Graph with Dynamic Edges Incident to a Designated Vertex
We consider an edge-weighted graph G with a designated vertex v0 such that weights of edges incident to v0 may increase or decrease. We show that, with an O(mn + n2 log n) time preprocessing, a minimum cut of the current G can be computed in O(log n) ...
Comments