skip to main content
10.1145/2505515.2505750acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Efficient processing of streaming graphs for evolution-aware clustering

Published: 27 October 2013 Publication History

Abstract

The clustering of vertices often evolves with time in a streaming graph, where graph update events are given as a stream of edge (vertex) insertions and deletions. Although a sliding window in stream processing naturally captures some cluster evolution, it alone may not be adequate, especially if the window size is large and the clustering within the windowed stream is unstable. Prior graph clustering approaches are mostly insensitive to clustering evolution. In this paper, we present an efficient approach to processing streaming graphs for evolution-aware clustering (EAC) of vertices. We incrementally manage individual connected components as clusters subject to a constraint on the maximal cluster size. For each cluster, we keep the relative recency of edges in a sorted order and favor more recent edges in clustering. We evaluate the effectiveness of EAC and compare it with a previous state-of-the-art evolution-insensitive clustering (EIC) approach. The results show that EAC is both effective and efficient in capturing evolution in a streaming graph. Moreover, we implement EAC as a streaming graph operator on IBM's InfoSphere Streams, a large-scale distributed middleware for stream processing, and show snapshots of the user cluster evolution in a streaming Twitter mention graph.

References

[1]
M. K. Agarwal, K. Ramamritham, and M. Bhide. Real time discovery of dense clusters in highly dynamic graphs: identifying real world events in highly dynamic environments. PVLDB, 5(10):980--991, 2012.
[2]
C. C. Aggarwal, J. Han, J. Wang, and P. S. Yu. A framework for clustering evolving data streams. In Proc. of the 29th Int. Conf. on Very Large Data Bases, 2003.
[3]
C. C. Aggarwal, Y. Zhao, and P. S. Yu. On clustering graph streams. In Proc. of the SIAM Int. Conf. on Data Mining, 2010.
[4]
A. Angel, N. Koudas, N. Sarkas, and D. Srivastava. Dense subgraph maintenance under streaming edge weight updates for real-time story indentification. PVLDB, 5(6):574--585, 2012.
[5]
B. Bahmani, R. Kumar, and S. Vassilvitskii. Densest subgraph in streaming and MapReduce. PVLDB, 5(5):454--465, 2012.
[6]
D. Ediger, R. McColl, J. Riedy, and D. A. Bader. STINGER: High performance data structure for streaming graphs. In Proc. of IEEE High Performance Extreme Computing Conf., 2012.
[7]
A. Eldawy, R. Khandekar, and K.-L. Wu. Clustering streaming graphs. In Proc. of the 32nd IEEE Int. Conf. on Distributed Computing Systems, 2012.
[8]
GraphInsight. http://www.graphinsight.com, 2013.
[9]
M. Gupta, C. C. Aggarwal, J. Han, and Y. Sun. Evolutionary clustering and analysis of bibliographic networks. In Proc. of Int. Conf. on Advances in Social Networks Analysis and Mining, 2011.
[10]
M. Henzinger and V. King. Randomized fully dynamic graph algorithm with polylogarithmic time per operation. Journal of the ACM, 46(4):502--516, 1999.
[11]
IBM. InfoSphere Streams. http://www.ibm.com/software/data/infosphere/streams/.
[12]
G. Karypis. Metis: a software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices. http://glaros.dtc.umn.edu/gkhome/views/metis, 2011.
[13]
V. Kawadia and S. Sreenivasan. Online detection of temporal communities in evolving networks by estrangement confinement. Nature Scientific Reports, 2012.
[14]
Y. Lin, Y. Chi, S. Zhu, H. Sundaram, and B. Tseng. Facetnet: a framework for analyzing communities and their evolutions in dynamic networks. In Proc. of the 17th Int. Conf. on World Wide Web, 2008.
[15]
I. Stanton and G. Kliot. Streaming graph partitioning for large distributed graphs. In Proc. of the 18th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, 2012.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge Management
October 2013
2612 pages
ISBN:9781450322638
DOI:10.1145/2505515
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. clustering streaming graphs
  2. evolution-aware clustering

Qualifiers

  • Research-article

Conference

CIKM'13
Sponsor:
CIKM'13: 22nd ACM International Conference on Information and Knowledge Management
October 27 - November 1, 2013
California, San Francisco, USA

Acceptance Rates

CIKM '13 Paper Acceptance Rate 143 of 848 submissions, 17%;
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2021)TripolineProceedings of the Sixteenth European Conference on Computer Systems10.1145/3447786.3456226(17-32)Online publication date: 21-Apr-2021
  • (2018)Real-Time Detection of Topics in Twitter StreamsEncyclopedia of Social Network Analysis and Mining10.1007/978-1-4939-7131-2_110157(2038-2063)Online publication date: 12-Jun-2018
  • (2017)KickStarterACM SIGARCH Computer Architecture News10.1145/3093337.303774845:1(237-251)Online publication date: 4-Apr-2017
  • (2017)KickStarterACM SIGPLAN Notices10.1145/3093336.303774852:4(237-251)Online publication date: 4-Apr-2017
  • (2017)KickStarterACM SIGOPS Operating Systems Review10.1145/3093315.303774851:2(237-251)Online publication date: 4-Apr-2017
  • (2017)KickStarterProceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3037697.3037748(237-251)Online publication date: 4-Apr-2017
  • (2017)Towards a Query-Less News Search Framework on TwitterDatabase Systems for Advanced Applications10.1007/978-3-319-55699-4_9(137-152)Online publication date: 22-Mar-2017
  • (2017)Real-Time Detection of Topics in Twitter StreamsEncyclopedia of Social Network Analysis and Mining10.1007/978-1-4614-7163-9_110157-1(1-26)Online publication date: 4-Sep-2017
  • (2016)Structural measures of clustering quality on graph samplesProceedings of the 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining10.5555/3192424.3192487(345-348)Online publication date: 18-Aug-2016
  • (2016)Synergistic Analysis of Evolving GraphsACM Transactions on Architecture and Code Optimization10.1145/299278413:4(1-27)Online publication date: 25-Oct-2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media