research-article

Public Access

DenseAlert: Incremental Dense-Subtensor Detection in Tensor Streams

Authors:

Christos FaloutsosAuthors Info & Claims

KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Pages 1057 - 1066

https://doi.org/10.1145/3097983.3098087

Published: 04 August 2017 Publication History

Abstract

Consider a stream of retweet events - how can we spot fraudulent lock-step behavior in such multi-aspect data (i.e., tensors) evolving over time? Can we detect it in real time, with an accuracy guarantee? Past studies have shown that dense subtensors tend to indicate anomalous or even fraudulent behavior in many tensor data, including social media, Wikipedia, and TCP dumps. Thus, several algorithms have been proposed for detecting dense subtensors rapidly and accurately. However, existing algorithms assume that tensors are static, while many real-world tensors, including those mentioned above, evolve over time.

We propose DENSESTREAM, an incremental algorithm that maintains and updates a dense subtensor in a tensor stream (i.e., a sequence of changes in a tensor), and DENSESALERT, an incremental algorithm spotting the sudden appearances of dense subtensors. Our algorithms are: (1) Fast and "any time": updates by our algorithms are up to a million times faster than the fastest batch algorithms, (2) Provably accurate: our algorithms guarantee a lower bound on the density of the subtensor they maintain, and (3) Effective: our DENSESALERT successfully spots anomalies in real-world tensors, especially those overlooked by existing algorithms.

References

[1]

Reid Andersen and Kumar Chellapilla 2009. Finding dense subgraphs with size bounds. WAW.

Digital Library

[2]

Brett W. Bader, Tamara G. Kolda, et al. 2017. MATLAB Tensor Toolbox Version 2.6. Available online. (2017).http://www.sandia.gov/~tgkolda/TensorToolbox/

[3]

Bahman Bahmani, Ashish Goel, and Kamesh Munagala. 2014. Efficient primal-dual graph algorithms for mapreduce WAW.

[4]

Bahman Bahmani, Ravi Kumar, and Sergei Vassilvitskii. 2012. Densest subgraph in streaming and mapreduce. PVLDB, Vol. 5, 5 (2012), 454--465.

Digital Library

[5]

Oana Denisa Balalau, Francesco Bonchi, TH Chan, Francesco Gullo, and Mauro Sozio. 2015. Finding subgraphs with maximum total density and limited overlap WSDM.

[6]

Alex Beutel, Wanhong Xu, Venkatesan Guruswami, Christopher Palow, and Christos Faloutsos 2013. Copycatch: stopping group attacks by spotting lockstep behavior in social networks WWW.

[7]

Sayan Bhattacharya, Monika Henzinger, Danupon Nanongkai, and Charalampos Tsourakakis 2015. Space-and time-efficient algorithm for maintaining dense subgraphs on one-pass dynamic streams STOC.

[8]

Loïc Cerf, Jérémy Besson, Céline Robardet, and Jean-Franccois Boulicaut. 2008. Data Peeler: Contraint-Based Closed Pattern Mining in n-ary Relations. SDM.

[9]

Moses Charikar. 2000. Greedy approximation algorithms for finding dense components in a graph. APPROX.

[10]

Gideon Dror, Noam Koenigstein, Yehuda Koren, and Markus Weimer 2012. The Yahoo! Music Dataset and KDD-Cup'11. In KDD Cup.

[11]

Alessandro Epasto, Silvio Lattanzi, and Mauro Sozio. 2015. Efficient densest subgraph computation in evolving graphs WWW.

[12]

Esther Galbrun, Aristides Gionis, and Nikolaj Tatti. 2016. Top-k overlapping densest subgraphs. Data Mining and Knowledge Discovery (2016), 1--32.

Digital Library

[13]

Andrew V Goldberg. 1984. Finding a maximum density subgraph. Technical Report.

[14]

Bryan Hooi, Hyun~Ah Song, Alex Beutel, Neil Shah, Kijung Shin, and Christos Faloutsos. 2016. FRAUDAR: Bounding Graph Fraud in the Face of Camouflage KDD.

Digital Library

[15]

Dmitry I Ignatov, Sergei O Kuznetsov, Jonas Poelmans, and Leonid E Zhukov 2013. Can triconcepts become triclusters? Int. J. General Systems Vol. 42, 6 (2013), 572--593.

[16]

Meng Jiang, Alex Beutel, Peng Cui, Bryan Hooi, Shiqiang Yang, and Christos Faloutsos. 2015. A general suspiciousness metric for dense blocks in multimodal data ICDM.

[17]

Meng Jiang, Peng Cui, Alex Beutel, Christos Faloutsos, and Shiqiang Yang 2014. Catchsync: catching synchronized behavior in large directed graphs KDD.

Digital Library

[18]

Samir Khuller and Barna Saha 2009. On finding dense subgraphs. ICALP.

Digital Library

[19]

Tamara G Kolda and Brett W Bader 2009. Tensor decompositions and applications. SIAM review, Vol. 51, 3 (2009), 455--500.

Digital Library

[20]

Richard P Lippmann, David J Fried, Isaac Graf, Joshua W Haines, Kristopher R Kendall, David McClung, Dan Weber, Seth E Webster, Dan Wyschogrod, Robert K Cunningham, et al. 2000. Evaluating intrusion detection systems: The 1998 DARPA off-line intrusion detection evaluation DISCEX.

[21]

Koji Maruhashi, Fan Guo, and Christos Faloutsos 2011. Multiaspectforensics: Pattern mining on large-scale heterogeneous networks with tensor analysis ASONAM.

[22]

Julian McAuley, Rahul Pandey, and Jure Leskovec. 2015. Inferring networks of substitutable and complementary products KDD.

Digital Library

[23]

Andrew McGregor, David Tench, Sofya Vorotnikova, and Hoa T Vu 2015. Densest subgraph in dynamic graph streams. In MFCS.

[24]

Alan Mislove, Massimiliano Marcon, Krishna P. Gummadi, Peter Druschel, and Bobby Bhattacharjee. 2007. Measurement and Analysis of Online Social Networks IMC.

[25]

B Aditya Prakash, Mukund Seshadri, Ashwin Sridharan, Sridhar Machiraju, and Christos Faloutsos. 2010. Eigenspokes: Surprising patterns and community structure in large graphs. PAKDD.

[26]

Barna Saha, Allison Hoch, Samir Khuller, Louiqa Raschid, and Xiao-Ning Zhang 2010. Dense subgraphs with restrictions and applications to gene annotation graphs RECOMB.

[27]

Jouni K Seppänen and Heikki Mannila 2004. Dense itemsets KDD.

[28]

Neil Shah, Alex Beutel, Brian Gallagher, and Christos Faloutsos 2014. Spotting suspicious link behavior with fbox: An adversarial perspective ICDM.

[29]

Kijung Shin, Tina Eliassi-Rad, and Christos Faloutsos. 2016. CoreScope: Graph Mining Using k-Core Analysis - Patterns, Anomalies and Algorithms ICDM.

[30]

Kijung Shin, Bryan Hooi, and Christos Faloutsos. 2016. M-Zoom: Fast Dense-Block Detection in Tensors with Quality Guarantees ECML/PKDD.

[31]

Kijung Shin, Bryan Hooi, Jisu Kim, and Christos Faloutsos. 2017. D-Cube: Dense-Block Detection in Terabyte-Scale Tensors WSDM.

[32]

Jimeng Sun, Dacheng Tao, and Christos Faloutsos 2006. Beyond streams and graphs: dynamic tensor analysis KDD.

Digital Library

[33]

Charalampos Tsourakakis, Francesco Bonchi, Aristides Gionis, Francesco Gullo, and Maria Tsiarli. 2013. Denser than the densest subgraph: extracting optimal quasi-cliques with quality guarantees KDD.

Digital Library

[34]

Shuo Zhou, Nguyen Xuan Vinh, James Bailey, Yunzhe Jia, and Ian Davidson 2016. Accelerating Online CP Decompositions for Higher Order Tensors KDD.

Cited By

Ma XLiu FWu JYang JXue SSheng Q(2025)Rethinking Unsupervised Graph Anomaly Detection With Deep Learning: Residuals and ObjectivesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.350130737:2(881-895)Online publication date: Feb-2025
https://doi.org/10.1109/TKDE.2024.3501307
Chen YJiang JSun SHe BChen M(2024)RUSH: Real-Time Burst Subgraph Detection in Dynamic GraphsProceedings of the VLDB Endowment10.14778/3681954.368202817:11(3657-3665)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.14778/3681954.3682028
Ekle OEberle W(2024)Anomaly Detection in Dynamic Graphs: A Comprehensive SurveyACM Transactions on Knowledge Discovery from Data10.1145/366990618:8(1-44)Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1145/3669906
Show More Cited By

Index Terms

DenseAlert: Incremental Dense-Subtensor Detection in Tensor Streams

Recommendations

D-Cube: Dense-Block Detection in Terabyte-Scale Tensors
WSDM '17: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining

How can we detect fraudulent lockstep behavior in large-scale multi-aspect data (i.e., tensors)? Can we detect it when data are too large to fit in memory or even on a disk? Past studies have shown that dense blocks in real-world tensors (e.g., social ...
Fast, Accurate, and Flexible Algorithms for Dense Subtensor Mining

Given a large-scale and high-order tensor, how can we detect dense subtensors in it? Can we spot them in near-linear time but with quality guarantees? Extensive previous work has shown that dense subtensors, as well as dense subgraphs, indicate ...
Non-negative sub-tensor ensemble factorization (NsTEF) algorithm. A new incremental tensor factorization for large data sets.

The design of a fast iterative algorithm for blind decomposition of tensors into subtensors,with a inspiration of learning method particularly adapted for large data set and high dimensions.The decomposing of the tensor in a set of subtensors in the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 2017

2240 pages

ISBN:9781450348874

DOI:10.1145/3097983

General Chairs:
Stan Matwin
Dalhousie University
,
Shipeng Yu
LinkedIn
,
Faisal Farooq
IBM

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 August 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

KDD '17

Sponsor:

KDD '17: The 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 13 - 17, 2017

NS, Halifax, Canada

Acceptance Rates

KDD '17 Paper Acceptance Rate 64 of 748 submissions, 9%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
864
Total Downloads

Downloads (Last 12 months)152
Downloads (Last 6 weeks)20

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ma XLiu FWu JYang JXue SSheng Q(2025)Rethinking Unsupervised Graph Anomaly Detection With Deep Learning: Residuals and ObjectivesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.350130737:2(881-895)Online publication date: Feb-2025
https://doi.org/10.1109/TKDE.2024.3501307
Chen YJiang JSun SHe BChen M(2024)RUSH: Real-Time Burst Subgraph Detection in Dynamic GraphsProceedings of the VLDB Endowment10.14778/3681954.368202817:11(3657-3665)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.14778/3681954.3682028
Ekle OEberle W(2024)Anomaly Detection in Dynamic Graphs: A Comprehensive SurveyACM Transactions on Knowledge Discovery from Data10.1145/366990618:8(1-44)Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1145/3669906
Jiang JChen YHe BChen MChen J(2024)Spade+: A Generic Real-Time Fraud Detection Framework on Dynamic GraphsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.339415536:11(7058-7073)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1109/TKDE.2024.3394155
Abd Majeed WHamad M(2024)Exploring High-Dimensional Outlier Detection: A Comprehensive Study on Methods and Applications Using PCA and k-NN Algorithm2024 21st International Multi-Conference on Systems, Signals & Devices (SSD)10.1109/SSD61670.2024.10548429(699-704)Online publication date: 22-Apr-2024
https://doi.org/10.1109/SSD61670.2024.10548429
Fang LFeng KGui JFeng SHu A(2023)Anonymous Edge Representation for Inductive Anomaly Detection in Dynamic Bipartite GraphProceedings of the VLDB Endowment10.14778/3579075.357908816:5(1154-1167)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.14778/3579075.3579088
Jo HBu FShin KFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Robust Graph Clustering via Meta Weighting for Noisy GraphsProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615038(1035-1044)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615038
Bhatia SWadhwa MKawaguchi KShah NYu PHooi BSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Sketch-Based Anomaly Detection in Streaming GraphsProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599504(93-104)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599504
Arenas MNair PRabbany RFarnadi GHorvát ÁHall WContractor NFröhling LOgnayova KTaneja HWeber IGligori? KMejova Y(2023)Social Media as a Vector for Escort Ads:A Study on OnlyFans advertisements on TwitterProceedings of the 15th ACM Web Science Conference 202310.1145/3578503.3583609(260-270)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3578503.3583609
Feng WLiu SCheng X(2023)Hierarchical Dense Pattern Detection in TensorsACM Transactions on Knowledge Discovery from Data10.1145/357702217:6(1-29)Online publication date: 28-Feb-2023
https://dl.acm.org/doi/10.1145/3577022
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten