skip to main content
10.1145/3578338.3593566acmconferencesArticle/Chapter ViewAbstractPublication PagesmetricsConference Proceedingsconference-collections
abstract
Public Access

Real-time Spread Burst Detection in Data Streaming

Published: 19 June 2023 Publication History

Abstract

Data streaming has many applications in network monitoring, web services, e-commerce, stock trading, social networks, and distributed sensing. This paper introduces a new problem of real-time burst detection in flow spread. It is a challenging problem because estimating flow spread requires us to remember all past data items and detecting bursts in real time requires us to minimize spread estimation overhead, which was not the priority in most prior work. This paper provides the first efficient, real-time solution for spread burst detection. It is designed based on a new real-time super spreader identifier, which outperforms the state of the art in terms of both accuracy and processing overhead. The super spreader identifier is in turn based on a new sketch design for real-time spread estimation, which outperforms the best existing sketches.

Supplemental Material

MKV File
In this video I present our paper titled "Real-time Spread Burst Detection in Data Streaming". Our paper has three sketch designs: SAS, RSI, and RBD. First, I introduce the concepts of network traffic measurement, sketches, flows, size and spread. Then, I present our first sketch, called Self-Adaptive Sketch (SAS), which estimates the spread of a single flow. Afterwards, I introduce our second sketch, Real-time Super spreader Identification (RSI), which is designed to estimate the spread of multiple flows and detect super spreaders (spreads that exceed a pre-set threshold). Finally, I introduce a new type of traffic metric, called spread burst. We design a sketch called Real-time Burst Detection (RBD), that uses RSI as a component to detect spread bursts, which are rapid increases and decreases in a flow's spread across several time intervals. Throughout the video I present our experimental results, which show better accuracy and real-time query capability for all of our sketches.

References

[1]
Ran Ben Basat, Xiaoqi Chen, Gil Einziger, Shir Landau Feibish, Danny Raz, and Minlan Yu. 2020. Routing Oblivious Measurement Analytics. In 2020 IFIP Networking Conference (Networking). IEEE, 449--457.
[2]
Philippe Flajolet, Éric Fusy, Olivier Gandouet, and Frédéric Meunier. 2007. Hyperloglog: The Analysis of a Near-optimal Cardinality Estimation Algorithm. In Discrete Mathematics and Theoretical Computer Science. Discrete Mathematics and Theoretical Computer Science, 137--156.
[3]
Amit Goyal, Hal Daumé III, and Graham Cormode. 2012. Sketch Algorithms for Estimating Point Queries in NLP. In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. 1093--1103.
[4]
Lu Tang, Qun Huang, and Patrick PC Lee. 2020. Spread Sketch: Toward Invertible and Network-wide Detection of Superspreaders. In INFOCOM. IEEE, 1608--1617.
[5]
Haibo Wang, Dimitrios Melissourgos, Chaoyi Ma, and Shigang Chen. 2023. Real-time Spread Burst Detection in Data Streaming. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 7, 2 (2023), 1--29.

Index Terms

  1. Real-time Spread Burst Detection in Data Streaming

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SIGMETRICS '23: Abstract Proceedings of the 2023 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems
      June 2023
      123 pages
      ISBN:9798400700743
      DOI:10.1145/3578338
      • cover image ACM SIGMETRICS Performance Evaluation Review
        ACM SIGMETRICS Performance Evaluation Review  Volume 51, Issue 1
        SIGMETRICS '23
        June 2023
        108 pages
        ISSN:0163-5999
        DOI:10.1145/3606376
        Issue’s Table of Contents
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 June 2023

      Check for updates

      Author Tags

      1. data stream
      2. real-time
      3. spread burst

      Qualifiers

      • Abstract

      Data Availability

      In this video I present our paper titled "Real-time Spread Burst Detection in Data Streaming". Our paper has three sketch designs: SAS, RSI, and RBD. First, I introduce the concepts of network traffic measurement, sketches, flows, size and spread. Then, I present our first sketch, called Self-Adaptive Sketch (SAS), which estimates the spread of a single flow. Afterwards, I introduce our second sketch, Real-time Super spreader Identification (RSI), which is designed to estimate the spread of multiple flows and detect super spreaders (spreads that exceed a pre-set threshold). Finally, I introduce a new type of traffic metric, called spread burst. We design a sketch called Real-time Burst Detection (RBD), that uses RSI as a component to detect spread bursts, which are rapid increases and decreases in a flow's spread across several time intervals. Throughout the video I present our experimental results, which show better accuracy and real-time query capability for all of our sketches. https://dl.acm.org/doi/10.1145/3578338.3593566#SIGMETRICS23-V7pmc035.mkv

      Funding Sources

      Conference

      SIGMETRICS '23
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 459 of 2,691 submissions, 17%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 104
        Total Downloads
      • Downloads (Last 12 months)68
      • Downloads (Last 6 weeks)12
      Reflects downloads up to 17 Jan 2025

      Other Metrics

      Citations

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media