Skip to main content
SpringerLink
Log in

Utilizing Complex Networks for Event Detection in Heterogeneous High-Volume News Streams

  • Conference paper
  • First Online:
Complex Networks and Their Applications VIII (COMPLEX NETWORKS 2019)

Part of the book series: Studies in Computational Intelligence ((SCI,volume 881))

Included in the following conference series:

Abstract

Detecting important events in high volume news streams is an important task for a variety of purposes. The volume and rate of online news increases the need for automated event detection methods that can operate in real time. In this paper we develop a network-based approach that makes the working assumption that important news events always involve named entities (such as persons, locations and organizations) that are linked in news articles. Our approach uses natural language processing techniques to detect these entities in a stream of news articles and then creates a time-stamped series of networks in which the detected entities are linked by co-occurrence in articles and sentences. In this prototype, weighted node degree is tracked over time and change-point detection used to locate important events. Potential events are characterized and distinguished using community detection on KeyGraphs that relate named entities and informative noun-phrases from related articles. This methodology already produces promising results and will be extended in future to include a wider variety of complex network analysis techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.kaggle.com/snapcrack/all-the-news.

  2. 2.

    http://www.conll.org.

References

  1. Aiello, L.M., Petkos, G., Martin, C., Corney, D., Papadopoulos, S., Skraba, R., Goker, A., Kompatsiaris, Y., Jaimes, A.: Sensing trending topics in Twitter. Trans. Multimedia 15(6), 1268–1282 (2013)

    Article  Google Scholar 

  2. Allan, J., Carbonell, J., Doddington, G., Yamron, J., Yang, Y.: Topic detection and tracking pilot study: final report. In: Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop (1998)

    Google Scholar 

  3. Wayne, C.: Topic detection and tracking (TDT): overview & perspective. In: Proceedings of the DARPA BNTUW (1998)

    Google Scholar 

  4. Allan, J., Lavrenko, V.: On-line new event detection and tracking. In: SIGIR98: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (1998)

    Google Scholar 

  5. Petrović, S., Osborne, M., Lavrenko, V.: Streaming first story detection with application to Twitter. In: HLT: Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 181–189. Association for Computational Linguistics, Stroudsburg (2010)

    Google Scholar 

  6. He, Q., Chang, K., Lim, E.P.: Analyzing feature trajectories for event detection. In: 30th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 207–214 (2007)

    Google Scholar 

  7. Weng, J., Lee, B.S.: Event detection in Twitter. In: International AAAI Conference on Web and Social Media (2011)

    Google Scholar 

  8. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  9. Li, H., Yamanishi, K.: Topic analysis using a finite mixture model. In: Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora (EMNLP), pp. 35–44 (2000)

    Google Scholar 

  10. Wartena, C., Brussee, R.: Topic detection by clustering keywords. In: Proceedings of the IEEE Computer Society DEXA Workshops, pp. 54–58 (2008)

    Google Scholar 

  11. Prabowo, R., Thelwall, M., Hellsten, I., Scharnhorst, A.: Evolving debates in online communication: a graph analytical approach. Internet Res. 18, 520–540 (2008)

    Article  Google Scholar 

  12. Fung, G.P.C., Yu, J.X., Yu, P.S., Lu, H.: Parameter free bursty events detection in text streams. In: 31st International Conference on Very Large Data Bases, pp. 181–192 (2005)

    Google Scholar 

  13. Popescu, A.M., Pennacchiotti, M., Paranjpe, D.: Extracting events and event descriptions from Twitter. In: 20th International Conference Companion on World Wide Web, pp. 105–106 (2011)

    Google Scholar 

  14. Melvin, S., Yu, W., Ju, P., Young, S., Wang, W.: Event detection and summarization using phrase network. In: Altun, Y., et al. (eds.) Machine Learning and Knowledge Discovery in Databases, vol. 10536. Springer, Cham (2017)

    Chapter  Google Scholar 

  15. Sayyadi, H., Hurst, M., Maykov, A.: Event detection and tracking in social streams. In: Proceedings of International AAAI Conference on Weblogs and Social Media (2009)

    Google Scholar 

  16. Blondel, V.D., Guillaume, J.-L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech. Theo. Exp. 2008, 10 (2008)

    MATH  Google Scholar 

  17. Finkel, J.R., Grenager, T. and Manning, C: Incorporating non-local information into information extraction systems by gibbs sampling. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistic, pp. 363–370 (2005)

    Google Scholar 

  18. Ritter, A., Clark, S., Mausam, Etzioni, O.: Named entity recognition in tweets: an experimental study. In: Conference on Empirical Methods in Natural Language Processing, pp. 1524–1534. Association for Computational Linguistics, Stroudsburg (2011)

    Google Scholar 

  19. El-Kishky, A., Song, Y., Wang, C.R., Han, J.: Scalable topical phrase mining from text corpora. Proc. VLDB Endowment 8, 305–316 (2014)

    Article  Google Scholar 

  20. O’Connor, B., Krieger, M., Ahn, D.: TweetMotif: exploratory search and topic summarization for Twitter. In: Cohen, W.W., Gosling, S., (eds.) ICWSM. The AAAI Press (2010)

    Google Scholar 

  21. Porter, M.F.: An algorithm for suffix stripping. In: Jones, K.S., Willett, P. (eds.) Readings in Information Retrieval, pp. 313–316. Morgan Kaufmann Publishers Inc., San Francisco (1997)

    Google Scholar 

  22. Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1986)

    MATH  Google Scholar 

  23. Xu, X., Yuruk, N., Feng, Z., Schweiger, T.A.J.: SCAN: a structural clustering algorithm for networks. In: KDD: 13th ACM International Conference on Knowledge Discovery and Data Mining, pp. 824–833. ACM, New York (2007)

    Google Scholar 

  24. Goethals, B.: Frequent set mining. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, pp. 377–397. Springer, Boston (2005)

    Chapter  Google Scholar 

  25. Murtagh, F.: A survey of recent advances in hierarchical clustering algorithms. Comput. J. 26(4), 354–359 (1983)

    Article  Google Scholar 

  26. Lloyd, S.P.: Least squares quantization in PCM. IEEE Trans. Inf. Theor 28(2), 129–137 (1982)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors acknowledge funding from a commercial entity, Adarga Ltd. (https://www.adarga.ai). The funder had no input or editorial influence over the manuscript.

Author information

Authors and Affiliations

  1. University of Exeter, North Park Road, Exeter, EX4 4QE, UK

    Iraklis Moutidis & Hywel T. P. Williams

Authors
  1. Iraklis Moutidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hywel T. P. Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Iraklis Moutidis .

Editor information

Editors and Affiliations

  1. University of Burgundy, Dijon Cedex, France

    Hocine Cherifi

  2. Università degli Studi di Milano, Milan, Italy

    Sabrina Gaito

  3. University of Aveiro, Aveiro, Portugal

    José Fernendo Mendes

  4. Universidad Carlos III de Madrid, Leganés, Madrid, Spain

    Esteban Moro

  5. Indiana University, Bloomington, IN, USA

    Luis Mateus Rocha

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Moutidis, I., Williams, H.T.P. (2020). Utilizing Complex Networks for Event Detection in Heterogeneous High-Volume News Streams. In: Cherifi, H., Gaito, S., Mendes, J., Moro, E., Rocha, L. (eds) Complex Networks and Their Applications VIII. COMPLEX NETWORKS 2019. Studies in Computational Intelligence, vol 881. Springer, Cham. https://doi.org/10.1007/978-3-030-36687-2_55

Download citation

Publish with us

Policies and ethics

Search

Navigation