Skip to main content
SpringerLink
Log in

Integrating multiple types of features for event identification in social images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the rapidly increasing popularity of social media sites, a large amount of user-generated data has been injected into the web. The data include a wide variety of real-world events. As a consequence, especially for social multimedia objects, it has become increasingly difficult to allow the browsing and organization of multimedia collections in a more effective manner. The approach we propose in this study addresses this problem, thus enabling the browsing and organization of multimedia collections in a natural way, i.e., by events. There have been some research studies on this problem. However, most of the previous approaches merge multiple types of features (e.g., textual content, visual content, user information and temporal information) of social media using a relatively simple mechanism. In this study, we merge multiple types of features in an integrated manner to identify the event associated with user-contributed social multimedia objects. We exploit the correlations between different types of features, i.e., textual content, visual content, user information and temporal information, to classify new social multimedia objects into their corresponding event categories. We accomplish this through a feature correlation graph (FCG) that uses features as nodes and the correlations among these features as edges for each event and individual multimedia object. We then employ a probabilistic model based on Markov random field to connect each new multimedia object with the correct event. We evaluate the algorithm on large-scale, real-world datasets of event images downloaded from Flickr, and the experimental results confirm the superiority of our approach over state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Available for download at http://www.mpiinf.mpg.de/yago-naga/yago/downloads.html

References

  1. Ah-Pine J, Bressan M, Clinchant S, Csurka G, Hoppenot Y, Renders J-M (2009) Crossing textual and visual content in different application scenarios. Multimed Tools Appl 42:31–56

    Article  Google Scholar 

  2. Amigo E, Gonzalo J, Artiles J, Verdejo F (2009) A comparison of extrinsic clustering evaluation metrics based on formal constraints. Inf Retr 12(4):461–486

    Article  Google Scholar 

  3. Bao B-K, Min W, Lu K, Xu C (2013) Social event detection with robust high-order co-clustering. In Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR)

  4. Becker H, Naaman M, Gravano L (2010) Learning similarity metrics for event identification in social media. In Proceeding of International Conference on Web Search and Data Mining (WSDM)

  5. Becker H, XiaoExploiting B, Naaman M, Gravano L (2010) Exploiting social links for event identification in social media. In Proceeding of the 3rd Annual Workshop on Search in Social Media (SSM)

  6. Blake A, Kohli P, Rother C (2011) Markov random fields for vision and image processing. The MIT Press, July

  7. Brenner M, Izquierdo E (2013) Social event detection and retrieval in collaborative photo collections. In Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR)

  8. Budanitsky A, Hirst G (2006) Evaluating WordNet-based measures of lexical semantic relatedness. J Comput Linguist 32(1):13–47

    Article  MATH  Google Scholar 

  9. Chen L, Roy A (2009) Event detection from flickr data through wavelet-based spatial analysis. In Proceeding of the Conference on Information and Knowledge Management (CIKM)

  10. Clinchant S, Ah-Pine J, Csurka G (2011) Semantic combination of textual and visual information in multimedia retrieval. In Proceedings of the 1st ACM International Conference on Multimedia Retrieval, ICMR ‘11

  11. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  12. Crandall D, Backstrom L, Huttenlocher D, Kleinberg J (2009) Mapping the world’s photos. In Proceeding of International World Wide Web Conferences (WWW)

  13. Cui B, Tung AKH, Zhang C, Zhao Z (2010) Multiple feature fusion for social media applications. In Proceeding of ACM Conference on Management of Data (SIGMOD)

  14. Feng H, Shi R, Chua T-S (2004) A bootstrapping framework for annotating and retrieving WWW images. In Proceedings of the 12th annual ACM International Conference on Multimedia (MM)

  15. Firan. CS, Georgescu M, Nejdl W, Paiu R (2010) Bringing order to your photos: event-driven classification of flickr images based on social knowledge. In Proceeding of the Conference on Information and Knowledge Management (CIKM)

  16. He X, Cai D, Wen J-R, Ma W-Y, Zhang H-J (2004) ImageSeer: clustering and searching WWW images using link and page layout analysis. Microsoft technical report, MSR-TR-2004-38

  17. Ji M, Han JW, Danilevsky M (2011) Ranking-based classification of heterogeneous information networks. In Proceeding of the ACM international conference on Knowledge discovery and data mining (SIGKDD)

  18. Jiang J, Conrath D (1997) Semantic similarity based on corpus statistics and lexical taxonomy. In Proceeding of International Conference on Research in Computational Linguistics (COLING)

  19. Kannan A, Talukdar PP, Rasiwasia N, Ke Q (2011) Improving product classification using images. In Proceeding of IEEE International Conference on Data Mining (ICDM)

  20. Kindermann R, Laurie Snell J (1980) Markov random fields and their applications. American mathematical society

  21. Kulkarni G, Premraj V, Dhar S, Li S, Choi Y, Berg A, Berg T (2011) Baby talk: understanding and generating simple image descriptions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  22. Kumaran G, Allan J (2004) Text classification and named entities for new event detection. In Proceeding of ACM International Conference on Information Retrieval (SIGIR)

  23. Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning

  24. Li Z, Wang B, Li M, Ma W-Y (2005) A probabilistic model for retrospective news event detection. In Proceeding of ACM International Conference on Information Retrieval (SIGIR)

  25. Liu H, Xie X, Tang X, Li Z-W, Ma W-Y (2004) Effective browsing of web image search results. In Proceedings of the ACM international workshop on Multimedia Information Retrieval (MIR)

  26. Lowd D, Domingos P (2005) Naive Bayes models for probability estimation. In Proceedings of the 22nd international conference on Machine learning

  27. Luo J, Yu J, Joshi D, Hao W (2008) Event recognition: viewing the world with a third eye. In Proceeding of ACM International Conference on Multimedia (MM)

  28. Metzler D, Bruce Croft W (2005) A markov random field model for term dependencies. In Proceeding of ACM International Conference on Information Retrieval (SIGIR)

  29. Pan J-Y, Yang H-J, Faloutsos C, Duygulu P (2004) Automatic multimedia cross-modal correlation discovery. In Proceeding of the ACM international conference on Knowledge discovery and data mining (SIGKDD)

  30. Ruocco M, Ramampiaro H (2012) A scalable algorithm for extraction and clustering of event-related pictures. Multimedia Tools and Applications Journal, pp 1–34. Springer. ISSN 1380–7501

  31. Shen Y, Fan JP (2010) Leveraging loosely-tagged images and inter-object correlations for tag recommendation. In Proceeding of ACM International Conference on Multimedia (MM)

  32. Strehl A, Ghosh J (2003) Cluster ensembles - a knowledge reuse framework for combining multiple partitions. J Mach Learn Res 3:583–617

    MathSciNet  MATH  Google Scholar 

  33. Suchanek FM, Kasneci G, Weikum G (2007) Yago: a core of semantic knowledge. In Proceeding of International World Wide Web Conferences (WWW)

  34. Topic detection and tracking evaluation. http://www.itl.nist.gov/iad/mig//tests/tdt/

  35. Wang X, Sun J-T, Chen Z, Zhai C (2006) Latent semantic analysis for multiple-type interrelated data objects. In Proceeding of ACM International Conference on Information Retrieval (SIGIR)

  36. Wu L, Li M, Li Z, Ma W-Y, Yu N (2007) Visual language modeling for image classification. In Proceeding of The international workshop on Multimedia Information Retrieval (MIR)

  37. Yang Y, Carbonell J, Brown R, Pierce T, Archibald BT, Liu X (1999) Learning approaches for detecting and tracking news events. IEEE Intell Syst Spec Issue Appl Intell Inf Retr 14(4):32–43

    Article  Google Scholar 

  38. Yang Y, Pierce T, Carbonell JG (1998) A study of retrospective and on-line event detection. In Proceeding of ACM International Conference on Information Retrieval (SIGIR)

  39. Zhang K, Zi J, Wu LG (2007) New Event Detection Based on Indexing-tree and Named Entity. In Proceeding of ACM International Conference on Information Retrieval (SIGIR)

  40. Znaidia A, Shabou A, Popescu A, le Borgne H, Hudelot C (2012) Multimodal feature generation framework for semantic image classification. In Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, ICMR ‘12, pp 38:1–38:8

  41. Zwol RV, Murdock V, Pueyo LG, Ramirez G (2008) Diversifying image search with user generated content. In Proceeding of the ACM international conference on Multimedia Information Retrieval (MIR)

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (NO. 61170189, NO.61202239 and NO. 61003111), the Fundamental Research Funds for the Central Universities, and the Opening Project of Beijing Key Laboratory of Internet Culture and Digital Dissemination Research (NO. ICDD201403).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Beihang University, Beijing, China

    Xiaoming Zhang, Zhoujun Li & Xiaoming Chen

  2. State Key Laboratory of Software Development Environment, Beihang University, Beijing, China

    Zhoujun Li

  3. Beijing Key Laboratory of Internet Culture and Digital Dissemination Research, Beijing, China

    Xueqiang Lv

Authors
  1. Xiaoming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhoujun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xueqiang Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoming Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Li, Z., Lv, X. et al. Integrating multiple types of features for event identification in social images. Multimed Tools Appl 75, 3301–3322 (2016). https://doi.org/10.1007/s11042-014-2436-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2436-x

Keywords

Search

Navigation