Weixiong Rao
Lei Chen
ABSTRACT
Massive graph analytics have become an important aspect of multiple diverse applications. With the growing scale of real world graphs, efficient execution of entire graph analytics has become a challenging problem. Recently a number of distributed graph processing systems (Pregel, PowerGraph, Trinity) and centralized systems (GraphChi and XStream) have been designed. Compared with high expense of distributed systems deployed on a cluster of commodity machines, the centralized systems on cheap PCs are very attractive propositions with low expense and comparable performance. By careful analysis, we find that (i) the graph computation abstraction in the centralized systems inherently adopted a batch model similar to the distributed systems. The batch model could lead to suboptimal performance. (ii) The execution model in the centralized systems advocates sequential operations on Solid State Disk (SSD) which are still slower than memory-based operations. In order to tackle the above efficiency issues in centralized systems, we first propose a novel continuous graph computation abstraction. This model continuously processes edges and updates computation results. It allows much faster convergence than the batch model. Second, we propose to maintain vertex states in memory and advocates memory-based operations for much faster I/O operations than sequential operations on SSD. Finally, we design an adaptive memory layout to minimize overall I/O cost. We develop a proof of concept prototype L-Graph and implement four example graph analytic applications atop L-Graph. Preliminary evaluation on real and synthetic graphs have verified that the proposed continuous model greatly performs the widely used batch model and L-Graph can achiever much higher efficiency than the state of arts GraphChi.
- J. E. Gonzalez, Y. Low, H. Gu, D. Bickson, and C. Guestrin. Powergraph: distributed graph-parallel computation on natural graphs. In Proceedings of the 10th USENIX conference on Operating Systems Design and Implementation (OSDI), pages 17--30. USENIX Association, 2012. Google Scholar
Digital Library
- A. Kyrola and G. Blelloch. Graphchi: Large-scale graph computation on just a PC. In Proceedings of the 10th conference on Symposium on Opearting Systems Design & Implementation (OSDI). USENIX Association, 2012. Google ScholarDigital Library
- Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein. Distributed graphlab: A framework for machine learning in the cloud. Proc. VLDB Endow., 5(8):716--727, 2012. Google ScholarDigital Library
- Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein. Distributed graphlab: A framework for machine learning in the cloud. PVLDB, 5(8):716--727, 2012. Google ScholarDigital Library
- A. Lumsdaine, D. Gregor, B. Hendrickson, and J. Berry. Challenges in parallel graph processing. Parallel Processing Letters, 17(1):5--20, 2007 2007.Google ScholarCross Ref
- G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: a system for large-scale graph processing. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pages 135--146. ACM, 2010. Google ScholarDigital Library
- A. Roy, I. Mihailovic, and W. Zwaenepoel. X-stream: Edge-centric graph processing using streaming partitions. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP), pages 472--488, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- B. Shao, H. Wang, and Y. Li. Trinity: A distributed graph engine on a memory cloud. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (SIGMOD), pages 505--516, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- G. Wang, W. Xie, A. J. Demers, and J. Gehrke. Asynchronous large-scale graph processing made easy. In CIDR, 2013.Google Scholar
- X. Zhu and Z. Ghahramani. Learning from labeled and unlabeled data with label propagation. Technical report, Technical Report Carnegie Mellon University-CALD-02--107, Carnegie Mellon University, 2002.Google Scholar
Index Terms
- L-Graph: A General Graph Analytic System on Continuous Computation
Recommendations
Collapsible subgraphs of a 4-edge-connected graph
AbstractJaeger in 1979 showed that every 4-edge-connected graph is supereulerian, graphs that have spanning eulerian subgraphs. Catlin in 1988 sharpened Jaeger’s result by showing that every 4-edge-connected graph is collapsible, graphs that ...
Traversability and connectivity of the middle graph of a graph
We define a graph M(G) as an intersection graph @W(F) on the point set V(G) of any graph G. Let X(G) be the line set of G and F = V'(G) @__ __ X(G), where V'(G) indicates the family of all one point subsets of the set V(G). Let M(G) = @W(F). M(G) is ...
Trivially noncontractible edges in a contraction critically 5-connected graph
An edge of a k-connected graph is said to be k-contractible if the contraction of the edge results in a k-connected graph. A k-connected graph with no k-contractible edge is said to be contraction critically k-connected. An edge of a k-connected graph ...
Comments