Skip to main content
SpringerLink
Log in

A survey of event analysis and mining from social multimedia

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent years, with the popularity of mobile devices and mobile Internet, more and more social media sites are growing in an explosive way. Therefore, the social hot event will be rapidly fermented by the interaction of a large number of network users, and a large amount of multimedia data (such as texts, images and videos) will be generated. Therefore, it is important and necessary to conduct the research of multimedia social event analysis to know the evolutionary trend of social event over time automatically. This paper provides a survey and summarizes major progresses in multimedia social event analysis. We focus on four areas: (1) multimedia social event representation; (2) multimedia social event detection and tracking; (3) multimedia social event evolutionary analysis; and (4) multimedia social event topic mining.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Ahmed A, Xing EP (2008) Dynamic non-parametric mixture models and the recurrent chinese restaurant process: with applications to evolutionary clustering. In: Siam International conference on data mining, SDM 2008, April 24-26, 2008, Atlanta, Georgia, USA, pp 219–230

  2. Allan J (2002) Detection as multi-topic tracking. Inf Retr 5(2-3):139–157

    Article  Google Scholar 

  3. Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. American Statistician 46(3):175–185

    MathSciNet  Google Scholar 

  4. Blei D, Jordan MI (2003) Modeling annotated data, 127–134

  5. Blei D, Mcauliffe JD (2010) Supervised topic models. Adv Neur Inf Process Syst 3:327–332

    Google Scholar 

  6. Blei D, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res Arch 3:993–1022

    MATH  Google Scholar 

  7. Boiman O, Shechtman E, Irani M (2008) In defense of nearest-neighbor based image classification. In: IEEE Conference on computer vision and pattern recognition, 2008. CVPR 2008, pp 1–8

  8. Caron F, Davy M, Doucet A (2008) Generalized polya urn for time-varying Dirichlet process mixtures

  9. Chen J, Yu J, Shen Y (2013) Towards topic trend prediction on a topic evolution model with social connection. In: Ieee/wic/acm International conferences on web intelligence and intelligent agent technology, pp 153–157

  10. Chen N, Liu Y, Zhang ZJ (2014) A forecasting system of micro-blog public opinion based on artificial neural network. In: 2014 Tenth international conference on intelligent information hiding and multimedia signal processing (IIH-MSP). IEEE, pp 868–871

  11. Chi Y, Song X, Zhou D, Hino K, Tseng BL (2007) Evolutionary spectral clustering by incorporating temporal smoothness pp 153–162

  12. Csurka G (2004) Visual categorization with bags of keypoints. Workshop Statist Learn Comput Vis Eccv 44(247):1–22

    Google Scholar 

  13. Das R, Zaheer M, Dyer C (2015) Gaussian LDA for topic models with word embeddings. In: Meeting of the association for computational linguistics and the international joint conference on natural language processing, pp 795–804

  14. Debole F, Sebastiani F (2004) Supervised term weighting for automated text categorization. Springer, Berlin

    Book  Google Scholar 

  15. Deerwester S (1990) Indexing by latent semantic indexing. Journal of the American Society Ofr Information Science, 41

  16. Diakopoulos N, Naaman M, Kivran-Swaine F (2010) Diamonds in the rough: social media visual analytics for journalistic inquiry. In: Visual analytics science and technology, pp 115–122

  17. Fang Y, Si L, Somasundaram N, Yu Z (2012) Mining contrastive opinions on political texts using cross-perspective topic model, pp 63–72

  18. Fang Q, Xu C, Sang J, Hossain MS, Muhammad G (2015) Word-of-mouth understanding: entity-centric multimodal aspect-opinion mining in social media. IEEE Trans Multimed 17(12):2281– 2296

    Article  Google Scholar 

  19. Firan CS, Georgescu M, Nejdl W, Paiu R (2010) Bringing order to your photos: event-driven classification of flickr images based on social knowledge. In: ACM International conference on information and knowledge management, pp 189–198

  20. Gao ZJ, Song Y, Liu S, Wang H, Wei H, Chen Y, Cui W (2012) Tracking and connecting topics via incremental hierarchical Dirichlet processes. In: IEEE International conference on data mining, pp 1056–1061

  21. Griffiths TL, Steyvers M (2004) Finding scientific topics. Proc Natl Acad Sci USA 101 Suppl 1(1):5228–5235

    Article  Google Scholar 

  22. Guillaumin M, Verbeek J, Schmid C (2010) Multimodal semi-supervised learning for image classification. In: Computer vision and pattern recognition, pp 902–909

  23. Haghighi A, Vanderwende L (2009) Exploring content models for multi-document summarization. In: Human language technologies: the 2009 conference of the North American chapter of the association for computational linguistics, pp 362–370

  24. Hardoon DR, Szedmak SR, Shawe-Taylor JR (2004) Canonical correlation analysis: an overview with application to learning methods. MIT Press

  25. Hofmann T (1999) Probabilistic latent semantic indexing. In: Proc Sigir, pp 50–57

  26. Hong R, Hu Z, Wang R, Wang M, Tao D (2016) Multi-view object retrieval via multi-scale topic models. IEEE Trans Image Process 25(12):5814–5827

    Article  MathSciNet  Google Scholar 

  27. Hong R, Yang Y, Wang M, Hua XS (2015) Learning visual semantic relationships for efficient visual retrieval. IEEE Trans Big Data 1(4):152–161

    Article  Google Scholar 

  28. Hong R, Zhang L, Zhang C, Zimmermann R (2016) Flickr circles: aesthetic tendency discovery by multi-view regularized topic modeling. IEEE Trans Multimed 18 (8):1555–1567

    Article  Google Scholar 

  29. Iwata T, Watanabe S, Yamada T, Ueda N (2009) Topic tracking model for analyzing consumer purchase behavior. In: International jont conference on artifical intelligence, pp 1427–1432

  30. Iwata T, Yamada T, Sakurai Y, Ueda N (2010) Online multiscale dynamic topic models. In: Proc. ACM SIGKDD international conference on knowl-edge discovery and data mining, pp 663–672

  31. Jelodar H, Wang Y, Yuan C, Feng X (2017) Latent Dirichlet allocation (LDA) and topic modeling: models, applications a survey

  32. Jin W, Srihari RK (2007) Graph-based text representation and knowledge discovery. In: ACM Symposium on applied computing, pp 807–811

  33. Joachims T (1999) Text categorization with support vector machines. In: Proc. of European conference on machine learning

  34. Kalamaras I, Drosou A, Tzovaras D (2014) Multi-objective optimization for multimodal visualization. IEEE Trans Multimed 16(5):1460–1472

    Article  Google Scholar 

  35. Kasiviswanathan SP, Melville P, Banerjee A, Sindhwani V (2011) Emerging topic detection using dictionary learning, 745–754

  36. Keller KL (1993) Conceptualizing, measuring and managing customer based brand equity. J Mark 57(1):1–22

    Article  Google Scholar 

  37. Kumaran G, Allan J (2004) Text classification and named entities for new event detection. In: Proc. of the ACM Sigir’04 conference, pp 297–304

  38. Lewis DD (1998) Naive (bayes) at forty: the independence assumption in information retrieval. In: European conference on machine learning, pp 4–15

  39. Li P, Yan Y, Wang C, Ren Z, Cong P, Wang H, Feng J (2016) Customer voice sensor: a comprehensive opinion mining system for call center conversation. In: IEEE International conference on cloud computing and big data analysis, pp 324–329

  40. Liu B, Zhang L (2012) A survey of opinion mining and sentiment analysis. Springer, US

    Book  Google Scholar 

  41. Liu J, Zha ZJ, Tian Q, Liu D, Yao T, Ling Q, Mei T (2016) Multi-scale triplet cnn for person re-identification. In: Proceedings of the 2016 ACM on multimedia conference. ACM, pp 192–196

  42. Makkonen J, Ahonen-Myka H, Salmenkivi M (2004) Simple semantics in topic detection and tracking. Inf Retr 7(3-4):347–368

    Article  Google Scholar 

  43. Marcombes P, Dalalyan A (2010) Towards optimal naive bayes nearest neighbor. In: European conference on computer vision, pp 171–184

  44. Maron ME (1961) Automatic indexing: an experimental inquiry. J Acm 8 (3):404–417

    Article  Google Scholar 

  45. Mccallum A (1998) A comparison of event models for naive bayes text classification. In: Proc. AAAI-98 workshop on learning for text categorization, pp 41–48

  46. Merler M, Huang B, Xie L, Hua G, Natsev A (2012) Semantic model vectors for complex video event recognition. IEEE Trans Multimed 14(1):88–101

    Article  Google Scholar 

  47. Moghaddam S, Ester M (2011) ILDA:interdependent LDA model for learning latent aspects and their ratings from online product reviews, pp 665–674

  48. Moghaddam S, Ester M (2012) On the design of LDA models for aspect-based opinion mining, pp 803–812

  49. Ngiam J, Khosla A, Kim M, Nam J, Lee H, Ng AY (2011) Multimodal deep learning. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 689–696

  50. Pan CC, Mitra P (2011) Event detection with spatial latent Dirichlet allocation. In: International ACM/IEEE joint conference on digital libraries, pp 349–358

  51. Putthividhy D, Attias HT, Nagarajan SS (2010) Topic regression multi-modal latent Dirichlet allocation for image annotation. In: Computer vision and pattern recognition, pp 3408–3415

  52. Qian S, Zhang T, Hong R, Xu C (2015) Cross-domain collaborative learning in social multimedia. In: ACM International conference on multimedia, pp 99–108

  53. Qian S, Zhang T, Xu C (2015) Boosted multi-modal supervised latent Dirichlet allocation for social event classification. Acm Trans Multimed Comput Commun Appl 11(2):27

    Article  Google Scholar 

  54. Qian S, Zhang T, Xu C, Shao J (2016) Multi-modal event topic model for social event analysis. IEEE Trans Multimed 18(2):233–246

    Article  Google Scholar 

  55. Qiu M, Jiang J (2013) A latent variable model for viewpoint discovery from threaded forum posts. In: NAACL

  56. Ramage D, Heymann P, Manning CD, Garcia-Molina H (2009) Clustering the tagged web, 54–63

  57. Rasiwasia N, Pereira JC, Coviello E, Doyle G, Lanckriet GRG, Levy R, Vasconcelos N (2010) A new approach to cross-modal multimedia retrieval. In: International conference on multimedia, pp 251–260

  58. Ren L, Dunson DB, Carin L (2008) The dynamic hierarchical Dirichlet process. In: International conference, pp 824–831

  59. Salton G (1974) A vector space model for automatic indexing. Commun Acm 18 (11):613–620

    Article  MathSciNet  Google Scholar 

  60. Sang J, Xu C (2012) Right buddy makes the difference: an early exploration of social relation analysis in multimedia applications. In: ACM International conference on multimedia, pp 19–28

  61. Sang J, Xu C, Jain R (2017) Social multimedia ming: from special to general. In: IEEE International symposium on multimedia, pp 481–485

  62. Sebastiani F (2002) Machine learning in automated text categorization. Acm Comput Surv 34(1):1–47

    Article  Google Scholar 

  63. Srivastava N, Salakhutdinov RR (2012) Multimodal learning with deep Boltzmann machines. In: Advances in neural information processing systems, pp 2222–2230

  64. Theil H, Chung CF (1988) Relations between two sets of variates: the bits of information provided by each variate in each set. Statist Probab Lett 6(3):137–139

    Article  MathSciNet  Google Scholar 

  65. Wang X, Mohanty N, Mccallum A (2005) Group and topic discovery from relations and text. In: Conference on statistical network analysis, pp 28–35

  66. Wan L, Zhu L, Fergus R (2012) A hybrid neural network-latent topic model, 1287–1294

  67. Wang M, Li H, Tao D, Lu K, Wu X (2012) Multimodal graph-based reranking for web image search. IEEE Trans Image Process 21(11):4649–4661

    Article  MathSciNet  Google Scholar 

  68. Wang C, Blei D, Heckerman D (2012) Continuous time dynamic topic models. Uai, 579–586

  69. Wang H, Zhang C, Yin H, Wang W, Zhang J, Xu F (2016) A unified framework for fine-grained opinion mining from online reviews. In: Hawaii International conference on system sciences, pp 1134–1143

  70. Wang M, Fu W, Hao S, Tao D, Wu X (2016) Scalable semi-supervised learning by efficient anchor graph regularization. IEEE Trans Knowl Data Eng 28 (7):1864–1877

    Article  Google Scholar 

  71. Wang M, Fu W, Hao S, Liu H, Wu X (2017) Learning on big graph: label inference and regularization with anchor hierarchy. IEEE Trans Knowl Data Eng 29(5):1101–1114

    Article  Google Scholar 

  72. Wu X, Ngo CW, Hauptmann AG (2008) Multimodal news story clustering with pairwise visual near-duplicate constraint. IEEE Trans Multimed 10(2):188–199

    Article  Google Scholar 

  73. Xu C, Xu C, Xu C (2016) Multi-modal multi-view topic-opinion mining for social event analysis. In: ACM on multimedia conference, pp 2–11

  74. Yang Y, Zhang J, Carbonell J, Jin C (2002) Topic-conditioned novelty detection. In: Eighth ACM SIGKDD international conference on knowledge discovery and data mining, pp 688–693

  75. Yang X, Zhang T, Xu C (2014) Cross-domain feature learning in multimedia. IEEE Trans Multimed 17(1):64–78

    Article  Google Scholar 

  76. Yu J, Cong Y, Qin Z, Wan T (2012) Cross-modal topic correlations for multimedia retrieval. In: International conference on pattern recognition, pp 246–249

  77. Zha ZJ, Hua XS, Mei T, Wang J, Qi GJ, Wang Z (2008) Joint multi-label multi-instance learning for image classification. In: IEEE Conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE, pp 1–8

  78. Zhang H, Zhuang Y, Wu F (2007) . Cross-modal correlation learning for clustering on image-audio dataset 40(8):273–276

    Google Scholar 

  79. Zhang J, Song Y, Zhang C, Liu S (2010) Evolutionary hierarchical Dirichlet processes for multiple correlated time-varying corpora. In: ACM SIGKDD International conference on knowledge discovery and data mining, Washington, Dc, Usa, July, pp 1079–1088

  80. Zhu J, Chen N, Perkins H, Zhang B (2013) Gibbs max-margin topic models with data augmentation. J Mach Learn Res 15(1):1073–1110

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 61772170), and the National Key Research and Development Program of China (No. 2017YFB0803301).

Author information

Authors and Affiliations

  1. Hefei University of Technology, 485, Danxia Road, Hefei City, People’s Republic of China

    Tianpeng Liu, Feng Xue, Jian Sun & Xiao Sun

Authors
  1. Tianpeng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feng Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feng Xue.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, T., Xue, F., Sun, J. et al. A survey of event analysis and mining from social multimedia. Multimed Tools Appl 79, 33431–33448 (2020). https://doi.org/10.1007/s11042-019-7567-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7567-7

Keywords

Search

Navigation