skip to main content
10.1145/2501221.2501235acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

An architecture for detecting events in real-time using massive heterogeneous data sources

Published: 11 August 2013 Publication History

Abstract

The wealth of information that is readily available nowadays grants researchers and practitioners the ability to develop techniques and applications that monitor and react to all sorts of circumstances: from network congestions to natural catastrophies. Therefore, it is no longer a question of whether this can be done, but how to do it in real-time, and if possible proactively. Consequently, it becomes a necessity to develop a platform that will aggregate all the necessary information and will orchestrate it in the best way possible, towards meeting these goals. A main problem that arises in such a setting is the high diversity of the incoming data, obtained from very different sources such as sensors, smart phones, GPS signals and social networks. The large volume of the incoming data is a gift that ensures high quality of the produced output, but also a curse, because higher computational resources are needed. In this paper, we present the architecture of a framework designed to gather, aggregate and process a wide range of sensory input coming from very different sources. A distinctive characteristic of our framework is the active involvement of citizens. We guide the description of how our framework meets our requirements through two indicative use cases.

References

[1]
Flood watch -- decision support system for real-time forecasting. Available online from: http://www.dhigroup.com/upload/publications/mike11/Skotner_MIKE_FLOOD_watch.pdf, Accessed on June 1st 2013.
[2]
Mike flood. Available online from: http://mikebydhi.com/Applications/CoastAndSea/CoastalFlooding.aspx, Accessed on June 1st 2013.
[3]
M. Ankerst, M. M. Breunig, H.-P. Kriegel, and J. Sander. Optics: ordering points to identify the clustering structure. In SIGMOD, Philadelphia, PA, June 1999.
[4]
H. Becker, D. Iter, M. Naaman, and L. Gravano. Identifying content for planned events across social media sites. In WSDM, 2012.
[5]
H. Becker, M. Naaman, and L. Gravano. Learning similarity metrics for event identification in social media. WSDM, 2010.
[6]
C. Costa, C. Laoudias, D. Zeinalipour-Yazti, and D. Gunopulos. Smarttrace: Finding similar trajectories in smartphone networks without disclosing the traces. In ICDE, pages 1288--1291, 2011.
[7]
J. Eisenstein, B. O'Connor, N. A. Smith, and E. P. Xing. A latent variable model for geographic lexical variation. In EMNLP, 2010.
[8]
A. J. G. Gray, J. Sadler, O. Kit, K. Kyzirakos, M. Karpathiotakis, J.-P. Calbimonte, K. Page, R. García-Castro, A. Frazer, I. Galpin, A. A. A. Fernandes, N. W. Paton, O. Corcho, M. Koubarakis, D. D. Roure, K. Martinez, and A. Gómez-Pérez. A semantic sensor web for environmental decisionăsupport applications. Sensors, 11(9):8855--8887, 2011.
[9]
M. Gupta, P. Zhao, and J. Han. Evaluating event credibility on twitter. In SDM, pages 153--164, 2012.
[10]
L. Hong, A. Ahmed, S. Gurumurthy, A. J. Smola, and K. Tsioutsiouliklis. Discovering geographical topics in the twitter stream. WWW, 2012.
[11]
M. L. V. Martina, E. Todini, and A. Libralon. A bayesian decision approach to rainfall thresholds based flood warning. Hydrology and Earth System Sciences, 10(3):413--426, 2006.
[12]
M. Mathioudakis and N. Koudas. Twittermonitor: trend detection over the twitter stream. In SIGMOD, 2010.
[13]
I. Mpoutsis, V. Kalogeraki, and D. Gunopulos. Efficient event detection by exploiting crowds. In The 7th ACM International Conference on Distributed Event-Based Systems (DEBS 2013), Arlington, Texas, USA, June-July 2013.
[14]
T. Sakaki, M. Okazaki, and Y. Matsuo. Earthquake shakes twitter users: real-time event detection by social sensors. In WWW, 2010.
[15]
J. Sander, M. Ester, H.-P. Kriegel, and X. Xu. Density-based clustering in spatial databases: The algorithm gdbscan and its applications. Data Min. Knowl. Discov., 2(2):169--194, June 1998.
[16]
J. Sutton, L. Palen, and I. Shlovski. Back-channels on the front lines: Emerging use of social media in the 2007 southern california wildfires. 2008.
[17]
G. Valkanas and D. Gunopulos. Location extraction from social networks with commodity software and online data. In ICDM Workshops (SSTDM), 2012.
[18]
G. Valkanas and D. Gunopulos. A ui prototype for emotion-based event detection in the live web. In SS-KDD-HCI @ SouthCHI, 2013.
[19]
J. Weng and B.-S. Lee. Event detection in twitter. In ICWSM, 2011.

Cited By

View all
  • (2019)Integration and Exploitation of Sensor Data in Smart Cities through Event-Driven ApplicationsSensors10.3390/s1906137219:6(1372)Online publication date: 19-Mar-2019
  • (2019)What's Happening Around the World? A Survey and Framework on Event Detection Techniques on TwitterJournal of Grid Computing10.1007/s10723-019-09482-217:2(279-312)Online publication date: 1-Jun-2019
  • (2016)Detecting Events in Online Social Networks: Definitions, Trends and ChallengesSolving Large Scale Learning Tasks. Challenges and Algorithms10.1007/978-3-319-41706-6_2(42-84)Online publication date: 3-Jul-2016
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
BigMine '13: Proceedings of the 2nd International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications
August 2013
119 pages
ISBN:9781450323246
DOI:10.1145/2501221
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 August 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. architecture
  2. event detection & response
  3. real-time

Qualifiers

  • Research-article

Funding Sources

  • EU
  • Greek National funds

Conference

KDD' 13
Sponsor:

Acceptance Rates

BigMine '13 Paper Acceptance Rate 13 of 23 submissions, 57%;
Overall Acceptance Rate 13 of 23 submissions, 57%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2019)Integration and Exploitation of Sensor Data in Smart Cities through Event-Driven ApplicationsSensors10.3390/s1906137219:6(1372)Online publication date: 19-Mar-2019
  • (2019)What's Happening Around the World? A Survey and Framework on Event Detection Techniques on TwitterJournal of Grid Computing10.1007/s10723-019-09482-217:2(279-312)Online publication date: 1-Jun-2019
  • (2016)Detecting Events in Online Social Networks: Definitions, Trends and ChallengesSolving Large Scale Learning Tasks. Challenges and Algorithms10.1007/978-3-319-41706-6_2(42-84)Online publication date: 3-Jul-2016
  • (2015)Temporal Aspects of Big Data ManagementProceedings of the 2015 22nd International Symposium on Temporal Representation and Reasoning (TIME)10.1109/TIME.2015.31(180-185)Online publication date: 23-Sep-2015
  • (2014)Spatial and temporal analysis of TwitterProceedings of the Conference on Principles, Systems and Applications of IP Telecommunications10.1145/2670386.2670392(1-6)Online publication date: 1-Oct-2014
  • (2014)Mining Twitter Data with Resource ConstraintsProceedings of the 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 0110.1109/WI-IAT.2014.29(157-164)Online publication date: 11-Aug-2014
  • (2014)Micro Analysis of Urban Vehicular Data for Enhanced Information Services for Commuters2014 IEEE 79th Vehicular Technology Conference (VTC Spring)10.1109/VTCSpring.2014.7023167(1-7)Online publication date: May-2014

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media