Skip to main content
SpringerLink
Log in

Event analysis in social multimedia: a survey

  • Review Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Recent years have witnessed the rapid growth of social multimedia data available over the Internet. The age of huge amount of media collection provides users facilities to share and access data, while it also demands the revolution of data management techniques, since the exponential growth of social multimedia requires more scalable, effective and robust technologies to manage and index them. The event is one of the most important cues to recall people’s past memory. The reminder value of an event makes it extremely helpful in organizing data. The study of event based analysis on social multimedia data has drawn intensive attention in research community. In this article, we provide a comprehensive survey on event based analysis over social multimedia data, including event enrichment, detection, and categorization. We introduce each paradigm and summarize related research efforts. In addition, we also suggest the emerging trends in this research area.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Mor N. Social Multimedia: highlighting opportunities for search and mining of multimedia aata in social media applications. Multimedia Tools and Applications, 2012, 56(1): 9–34

    Article  Google Scholar 

  2. Luo J B, Dhiraj J, Yu J, Andrew G. Geotagging in multimedia and computer vision—a survey. Multimedia Tools and Applications, 2011, 51(1): 187–211

    Article  Google Scholar 

  3. Alessandro V, Maja P, Herve B. Social signal processing: survey of an emerging domain. Image and Vision Computing, 2009, 27(12): 1743–1759

    Article  Google Scholar 

  4. Mei T, Rui Y, Li S P, Tian Q. Multimedia search reranking: a literature survey. ACM Computing Surveys, 2014, 46(3)

    Article  Google Scholar 

  5. Wang M, Ni B B, Hua X S, Tat-Seng C. Assistive tagging: a survey of multimedia tagging with human-computer joint exploration. ACM Computing Surveys, 2012, 44(4)

    Article  Google Scholar 

  6. Yang Q. Three challenges in data mining. Frontiers of Computer Science in China, 2010, 4(3): 324–333

    Article  Google Scholar 

  7. Ma H X, Qian WN, Xia F, He X F, Xu J, Zhou A Y. Towards modeling popularity of microblogs. Frontiers of Computer Science, 2013, 7(2): 171–184

    Article  MathSciNet  Google Scholar 

  8. Utz W, Ramesh J. Toward a common event model for multimedia applications. IEEE MultiMedia, 2007, 14(1): 19–29

    Article  Google Scholar 

  9. Liu X L, Raphaël T, Benoit H. Finding media illustrating events. In: Proceedings of ACM International Conference on Multimedia Retrieval, 2011

    Google Scholar 

  10. Scherp A, Jain R, Kankanhalli M, Mezaris V. Modeling, detecting, and processing events in multimedia. In: Proceedings of the International Conference on Multimedia, 2010, 1739–1740

    Google Scholar 

  11. Petkos G, Papadopoulos S, Mezaris V, Troncy R, Cimiano P, Reuter T, Kompatsiaris Y. Social event detection at mediaeval: a threeyear retrospect of tasks and results. In: Proceedings of ACM ICMR 2014 Workshop on Social Events in Web Multimedia, 2014

    Google Scholar 

  12. Reuter T, Papadopoulos S, Petkos G, Mezaris V, Kompatsiaris Y, Cimiano P, de Vries C, Geva S. Social event detection at mediaeval 2013: challenges, datasets, and evaluation. In: Proceedings of the MediaEval 2013 Multimedia Benchmark Workshop, 2013

    Google Scholar 

  13. Troncy R, Malocha B, Fialho A T S. Linking events with media. In: Proceedings of the 6th International Conference on Semantic Systems, 2010

    Google Scholar 

  14. Becker H, Naaman M, Gravano L. Learning similarity metrics for event identification in social media. In: Proceedings of the 3rd ACM International Conference on Web Search and Data Mining. 2010, 291–300

    Chapter  Google Scholar 

  15. Liu X L, Troncy, R Huet B. Usingsocial media to identifyevents. In Proceedings of the ACM SIGMM Workshop on Social Media. 2011, 3–8

    Chapter  Google Scholar 

  16. Trabelsi C, Yahia S B. A probabilistic approach for events identification from social media RSS feeds. Database Systems for Advanced Applications, 2013, 7827: 139–152

    Article  Google Scholar 

  17. Firan C S, Georgescu M, Nejdl W, Paiu R. Bringing order to your photos: event-driven classification of Flickr images based on social knowledge. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, 2010, 189–198

    Google Scholar 

  18. Liu X L, Huet B. Heterogeneous features and model selection for event-based media classification. In: Proceedings of International Conference on Multimedia Retrieval. 2013, 151–158

    Chapter  Google Scholar 

  19. Billsus D, Pazzani M J. A hybrid user model for news story classification. In: Proceedings of the 7th International Conference on User Modeling. 1999, 99–108

    Google Scholar 

  20. Toda H, Kataoka R. A clustering method for news articles retrieval system. In: Proceedings of the International Conference on World Wide Web. 2005

    Google Scholar 

  21. Delgado D, Magalhães J, Correia N. Assisted news reading with automated illustrations. In: Proceedings of ACM Conference on Multimedia. 2010, 1647–1650

    Google Scholar 

  22. Joshi D, Wang J Z, Jia L. The story picturing engine—a system for automatic text illustration. ACM Transactions on Multimedia Computing Communications and Applications, 2006, 2(1): 68–89

    Article  Google Scholar 

  23. Cooper M, Foote J, Girgensohn A, Wilcox L. Temporal event clustering for digital photo collections. In: Proceedings of ACM International Conference on Multimedia, 2003

    Google Scholar 

  24. Graham A, Garcia-Molina H, Paepcke A, Winograd T. Time as essence for photo browsing through personal digital libraries. In: Proceedings of the ACM/IEEE-CS Joint Conference on Digital Libraries. 2002, 326–335

    Google Scholar 

  25. Jou B, Li H Z, Ellis J G, Morozoff-Abegauz D, Chang S F. Structured exploration of who, what, when, and where in heterogeneous multimedia news sources. In: Proceedings of the ACM International Conference on Multimedia. 2013, 357–360

    Chapter  Google Scholar 

  26. Kim M, Xie L X, Christen P. Event diffusion patterns in social media. In: Proceedings of the International AAAI Conference onWeblogs and Social Media. 2012

    Google Scholar 

  27. Burton K, Kasch N, Soboroff I. The icwsm 2011 Spinn3r dataset. In: Proceedings of the 5th Annual Conference on Weblogs and Social Media. 2011

    Google Scholar 

  28. Mattivi R, Uijlings J, De Natale F, Sebe N. Categorization of a collection of pictures into structured events. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012

    Google Scholar 

  29. Mattivi R, Uijlings J, De Natale F G B, Sebe N. Exploitation of Time Constraints for (sub-)Event Recognition. In: Proceedings of the 2011 Joint ACM Workshop on Modeling and Representing Events. 2011, 7–12

    Chapter  Google Scholar 

  30. Cooper M, Foote J, Girgensohn A, Wilcox L. Temporal event clustering for digital photo collections. ACM Transactions on Multimedia Computing, Communications, and Applications, 2005, 1(3): 269–288

    Article  Google Scholar 

  31. Sinha P, Jain R. Extractive summarization of personal photos from life events. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2011

    Google Scholar 

  32. Kennedy L, Naaman m. Less talk, more rock: automated organization of community-contributed collections of concert videos. In: Proceedings of the 18th ACM International Conference on World Wide Web. 2009, 311–320

    Chapter  Google Scholar 

  33. Diakopoulos N, Naaman M, Kivran-Swaine F. Diamonds in the rough: social media visual analytics for journalistic inquiry. In: Proceedings of 2010 IEEE Symposium on Visual Analytics Science and Technology. 2010, 115–122

    Chapter  Google Scholar 

  34. Gao M Y, Hua X S, Jain R. Wonder What: real-time event determination from photos. In: Proceedings of the 20th World Wide Web Conference. 2011

    Google Scholar 

  35. Liu X L, Huet B. Event representation and visualization from social media. Lecture Notes in Computer Science, 2013, 8294: 740–749

    Article  Google Scholar 

  36. Tang J L, Wang X F, Gao H J, Hu X, Liu H. Enriching short text representation in microblog for clustering. Frontiers of Computer Science, 2012, 6(1): 88–101

    MathSciNet  MATH  Google Scholar 

  37. Zheng X L, Zhong Y G, Daniel Z, Wang F Y. Social influence and spread dynamics in social networks. Frontiers of Computer Science, 2012, 6(5): 611–620

    MathSciNet  Google Scholar 

  38. Liu X L, Huet B. Automatic concept detector refinement for large-scale video semantic annotation. In: Proceedings of the 4th IEEE International Conference on Semantic Computing. 2010, 97–100

    Google Scholar 

  39. Bird S, Klein E, Loper E. Natural language processing with python. Sebastopol: O’Reilly Media, Inc., 2009

    MATH  Google Scholar 

  40. Auer S, Bizer C, Kobilarov G, Lehmann J, Ives Z. Dbpedia: a nucleus for a Web of open data. In: Proceedings of the 6th International Semantic Web Conference. 2007, 11–15

    Google Scholar 

  41. Sun C J, Guan Y. A statistical approach for content extraction from Web page. Journal of Chinese Information Processing, 18(5): 17–22, 2004

    Google Scholar 

  42. Mei T, Yang B, Yang S Q, Hua X S. Video collage: presenting a video sequence using a single image. The Visual Computer, 2009, 25(1): 39–51

    Article  Google Scholar 

  43. Aouiche K, Lemire D, Godin R. Web 2.0 OLAP: from data cubes to tag clouds. Lecture Notes in Business Information Processing, 2009

    Google Scholar 

  44. Quack T, Leibe B, Van Gool L. World-scale mining of objects and events from community photo collections. In: Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval, 2008, 47–56

    Chapter  Google Scholar 

  45. Papadopoulos S, Zigkolis C, Kompatsiaris Y, Vakali A. Cluster-based landmark and event detection for tagged photo collections. IEEE Multimedia, 2011, 18(1): 52–63

    Article  Google Scholar 

  46. Becker H, Naaman H, Gravano L. Event identification in social media. In: Proceedings of the 12th International Workshop on the Web and Databases. 2009

    Google Scholar 

  47. Petkos G, Papadopoulos S, Schinas E, Kompatsiaris Y. Graph-based multimodal clustering for social event detection in large collections of images. Lecture Notes in Computer Science, 2014, 8325: 146–158

    Article  Google Scholar 

  48. Petkos G, Papadopoulos S, Kompatsiaris Y. Social event detection using multimodal clustering and integrating supervisory signals. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012

    Google Scholar 

  49. Rattenbury T, Good N, Naaman M. Towards automatic extraction of event and place semantics from flickr tags. In: Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval. 2007, 103

    Google Scholar 

  50. Liu Z Y, Chen X X, Sun M S. Mining the interests of chinese microbloggers via keyword extraction. Frontiers of Computer Science, 2012, 6(1): 76–87

    MathSciNet  Google Scholar 

  51. Chen L, Abhishek R. Event detection from Flickr data through waveletbased spatial analysis. In: Proceedings of ACM conference on CIKM. 2009

    Google Scholar 

  52. Weng J S, Lee B S. Event detection in twitter. In: Proceedings of International AAAI Conference on Weblogs and Social Media. 2011

    Google Scholar 

  53. Deerwester S, Dumais S T, Furnas G W, Landauer T K, Harshman R. Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 1990, 41(6): 391–407

    Article  Google Scholar 

  54. Hofmann T. Probabilistic latent semantic indexing. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1999, 50–57

    Google Scholar 

  55. Blei D M, Ng A Y, Jordan M I. Latent Dirichlet Allocation. Journal of Machine Learning Research. 2003, 3(4–5): 993–1022

    MATH  Google Scholar 

  56. Pan C C, Mitra P. Event detection with spatial latent dirichlet allocation. In: Proceeding of the 11th Annual International ACM/IEEE joint Conference on Digital Libraries. 2011, 349

    Chapter  Google Scholar 

  57. Liu X L, Huet B. Social event discovery by topic inference. In: Proceeding of IEEE International Workshop on Image Analysis for Multimedia Interactive Services. 2012, 1–4

    Google Scholar 

  58. Wang X, Zhu F, Jiang J, Li S J. Real time event detection in twitter. Lecture Notes in Computer Science, 2013, 7923: 502–513

    Article  Google Scholar 

  59. The Y W, Jordan M I, Beal M J, Blei D M. Hierarchical dirichlet processes. Journal of the American Statistical Association, 2004, 101

    Google Scholar 

  60. Cheng T, Wicks T. Event detection using Twitter: a spatio-temporal approach. PLoS ONE, 2014, 9(6): e97807

    Article  Google Scholar 

  61. Reuter T, Cimiano P. Event-based classification of social media streams. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, 2012

    Google Scholar 

  62. Wang Y X, Sundaram H, Xie L X. Social event detection with interaction graph modeling. In: Proceedings of the 20th ACM International Conference on Multimedia, 2012, 865–868

    Chapter  Google Scholar 

  63. Brenner M, Izquierdo E. Social event detection and retrieval in collaborative photo collections. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012

    Google Scholar 

  64. Liu X L, Huet B, Troncy R. Eurecom mediaeval 2011 social event detection task. MediaEval, 2011

    Google Scholar 

  65. Li R H, Liu J Q, Yu J X, Chen H X, Kitagawa H. Co-occurrence prediction in a large location-based social network. Frontiers of Computer Science, 2013, 7(2): 185–194

    Article  MathSciNet  Google Scholar 

  66. Makkonen J, Ahonen-Myka H, Salmenkivi M. Simple semantics in topic detection and tracking. Information Retrieval, 2004, 3(7): 347–368

    Article  MATH  Google Scholar 

  67. Reuter T, Cimiano P. Event-based classification of social media streams. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012

    Google Scholar 

  68. Gu J M, Wu Y L, Hung W C, Tang C Y. Personal photo organization using event annotation. In: Proceedings of the 9th International Conference on Information, Communications and Signal Processing. 2013, 1–4

    Google Scholar 

  69. Qian S H, Zhang T Z, Xu C S. Multimodal supervised latent dirichlet allocation for event classification in social media. In: Proceedings of International Conference on Internet Multimedia Computing and Service, 2014, 152–157

    Google Scholar 

  70. Suchanek F M, Kasneci G, Weikum G. Yago: a large ontology from Wikipedia and WordNet. Web Semantics: Science, Services and Agents on the World Wide Web, 2008, 6(3): 203–217

    Article  Google Scholar 

  71. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten I H. The WEKA data mining software: an update. ACM SIGKDD Explorations Newsletter, 2009, 11(1): 10–18

    Article  Google Scholar 

  72. Chang C C, Lin C J. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 27:1–27

    Article  Google Scholar 

  73. Breiman L, Friedman J, Olshen R, Stone C. Classification and regression trees. Monterey: Wadsworth and Brooks, 1984

    MATH  Google Scholar 

  74. Breiman L. Random forests. Machine Learning, 2001, 45: 5–32

    Article  MATH  Google Scholar 

  75. Hays J, Efros A A. IM2GPS: estimating geographic information from a single image. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2008, 1–8

    Google Scholar 

  76. Lo H Y, Chang K W, Chen S T, Chiang S H, Ferng C S, Hsieh C J, Ko Y K, Kuo T T, Lai H C, Lin K Y, Wang C H, Yu H F, Lin C J, Lin H T, Lin S. An ensemble of three classifiers for KDD Cup 2009: expanded linear model, heterogeneous boosting, and selective naïve bayes. Journal of Machine Learning Research—Proceedings Track, 2009, 7: 57–64

    Google Scholar 

  77. Manning C D, Raghavan P, Schütze H. Introduction to information retrieval. Cambridge: Cambridge University Press, 2008

    Book  MATH  Google Scholar 

  78. Chua T S, Tang J H, Hong R C, Li H J, Luo Z P, Zheng Y T. Nus-wide: a real-world Web image database from national university of singapore. In: Proceedings of ACM Conferrence on Image and Video Retrieval. 2009

    Google Scholar 

  79. Hebeler J, Fisher M, Blace R, Perez-Lopez A. Semantic Web programming. Indianapolis: Wiley Publishing, 2009

    Google Scholar 

  80. Petkos G, Papadopoulos S, Mezaris V, Kompatsiaris Y. Social event detection at mediaeval 2014: challenges, datasets, and evaluation. MediaEval, 2013

    Google Scholar 

  81. Over P, Awad G, Michel M, Fiscus J, Sanders G, Kraaij W, Smeaton A F, Quvenot G. TRECVID 2014—an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID. 2010, 15–17

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer and Information, Hefei University of Technology, Hefei, 230009, China

    Xueliang Liu & Meng Wang

  2. Department of Multimedia, EURECOM sophia-antipolis, Nice, 06560, France

    Benoit Huet

Authors
  1. Xueliang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benoit Huet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xueliang Liu.

Additional information

Xueliang Liu received his MS and PhD from the University of Science and Technology of China, China and EURECOM, France in 2008 and 2012, respectively. He is currently with the School of Computing and Information, Hefei University of Technology. His current research interests include social media mining, multimodality modeling, and social event analysis.

Meng Wang received his BE and PhD in the Special Class for the Gifted Young and the Department of Electronic Engineering and Information Science from University of Science and Technology of China, China. He is a Professor of Hefei University of Technology, China. His current research interests include multimedia content analysis, search, mining, recommendation, and large-scale computing.

Benoit Huet received his PhD in computer science from University of York, UK. He is an assistant professor at the Multimedia Information Processing Group, EURECOM, France. His current research interests include largescale multimedia content analysis, mining and indexing, multimodal fusion, and socially-aware multimedia.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Wang, M. & Huet, B. Event analysis in social multimedia: a survey. Front. Comput. Sci. 10, 433–446 (2016). https://doi.org/10.1007/s11704-015-4583-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-015-4583-2

Keywords

Search

Navigation