Skip to main content
SpringerLink
Log in

On the automatic online collection of training data for visual event modeling

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The last decade has witnessed the development and uprising of social media web services. The use of these shared online media as a source of huge amount of data for research purposes is still a challenging problem. In this paper, a novel framework is proposed to collect training samples from online media data to model the visual appearance of social events automatically. The visual training samples are collected through the analysis of the spatial and temporal context of media data and events. While collecting positive samples can be achieved easily thanks to dedicated event machine-tags, finding the most representative negative samples from the vast amount of irrelevant multimedia documents is a more challenging task. Here, we argue and demonstrate that the most common negative samples, originating from the same location as the event to be modeled, are best suited for the task. A novel ranking approach is devised to automatically select a set of negative samples. Finally the automatically collected samples are used to learn visual event models using Support Vector Machine (SVM). The resulting event models are effective to filter out irrelevant photos and perform with a high accuracy as demonstrated on various social events originating for various categories of events.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://www.last.fm

  2. http://www.upcoming.org

  3. http://www.facebook.com/events/

  4. http://www.flickr.com

References

  1. Aggarwal JK, Cai Q (1997) Human motion analysis: a review. In: IEEE nonrigid and articulated motion workshop, pp 90–102

  2. Arase Y, Xie X, Hara T, Nishio S (2010) Mining people’s trips from large scale geo-tagged photos. In: 18th ACM international conference on multimedia (ACM MM’10). Firenze, Italy, pp 133–142

  3. Ballan L, Bertini M, Bimbo A, Seidenari L, Serra G (2010) Event detection and recognition for semantic annotation of video. Multimed Tools Appl 51(1):279–302

    Article  Google Scholar 

  4. Becker H, Iter D, Naaman M, Gravano L (2012) Identifying content for planned events across social media sites. In: ACM conference on WSDM

  5. Becker H, Naaman M, Gravano L (2009) Event identification in social media. In: 12th international workshop on the web and databases (WebDB’09). Providence, USA

  6. Becker H, Naaman M, Gravano L (2010) Learning similarity metrics for event identification in social media. In: 3rd ACM international conference on web search and data mining (WSDM’10). New York, USA, pp 291–300

  7. Bishop CM (2006) Pattern recognition and machine learning. Springer

  8. Chua T-S, Tang J, Hong R, Li H, Luo Z, Zheng Y-T (2009) NUS-WIDE: a real-world web image database from National University of Singapore. In: Proc. of ACM conf. on image and video retrieval. Santorini, Greece

  9. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv (CSUR) 40:5:1–5:60

    Article  Google Scholar 

  10. Delgado D, Magalhaes J, Correia N (2010) Automated illustration of news stories. In: 2010 IEEE fourth international conference on semantic computing. IEEE, pp 73–78

  11. Firan CS, Georgescu M, Nejdl W, Paiu R (2010) Bringing order to your photos: event-driven classification of flickr images based on social. In: Proceedings of the 19th ACM international conference on information and knowledge management—CIKM ’10. ACM Press, New York, New York, USA, p 189

  12. Hong R, Li G, Nie L, Tang J, Chua T-S (2010) Explore large scale data for multimedia QA. In: ACM conference on CIVR. Xi’an, China

  13. Kennedy L, Naaman M (2009) Less talk, more rock: automated organization of community-contributed collections of concert videos. In: 18th ACM international conference on world wide web (WWW’09). Madrid, Spain, pp 311–320

  14. Li L-J, Wang G (2007) OPTIMOL: automatic online picture collection via incremental model learning. IEEE Conference on CVPR 88(2):1–8

    Google Scholar 

  15. Li X, Snoek CG, Worring M, Smeulders AW (2011) Social negative bootstrapping for visual categorization. In: Proceedings of the ACM international conference on multimedia retrieval

  16. Liu D, Hua X-S, Wang M, Zhang H-J (2010) Image retagging. In: 18th ACM international conference on multimedia (ACM MM’10). Firenze, Italy, pp 491–500

  17. Liu T-Y (2011) Learning to rank for information retrieval. Springer

  18. Liu X, Troncy R, Huet B (2011) Finding media illustrating events. In: ACM conference on ICMR

  19. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  20. Manning CD, Raghavan P, Schütze H (2008) Introduction to information retrieval, 1st edn. Cambridge University Press

  21. Over P, Awad G, Michel M, Fiscus J, Kraaij W, Smeaton AF (2011) Trecvid 2011—an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID. NIST, USA

  22. Papadopoulos S, Schinas E, Mezaris V, Troncy R, Kompatsiaris I (2012) Social event detection at mediaeval 2012: challenges, dataset and evaluation. In: MediaEval’12, pp –1–1

  23. Quack T, Leibe B, Van Gool L (2008) World-scale mining of objects and events from community photo collections. In: ACM conference on CIVR. New York, USA, pp 47

  24. Schroff F, Criminisi A, Zisserman A (2007) Harvesting image databases from the web. In: IEEE 11th international conference on computer vision. IEEE, pp 1–8

  25. Tang J, Yan S, Hong R, Qi G-J, Chua T-S (2009) Inferring semantic concepts from community-contributed images and noisy tags. In: 17th ACM international conference on multimedia (ACM MM’09). Beijing, China, pp 223–232

  26. Wang Y, Sundaram H, Xie L (2012) Social event detection with interaction graph modeling. In: ACM conference on multimedia

  27. Zha Z-J, Mei T, Wang J, Wang Z, Hua X-S (2009) Graph-based semi-supervised learning with multi-label. ACM Trans Program Lang Syst 20(5):97–103

    Google Scholar 

  28. Zhang L, Lin F, Zhang B (2001) Support vector machine learning for image retrieval. IEEE Conference on ICIP 2(x):721–724

    Google Scholar 

Download references

Acknowledgements

The research leading to this paper was partially supported by the project AAL-2009-2-049 “Adaptable Ambient Living Assistant” (ALIAS) co-funded by the European Commission and the French Research Agency (ANR) in the Ambient Assisted Living (AAL) programme.

Author information

Authors and Affiliations

  1. EURECOM, Boit, France

    Xueliang Liu & Benoit Huet

Authors
  1. Xueliang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benoit Huet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xueliang Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, X., Huet, B. On the automatic online collection of training data for visual event modeling. Multimed Tools Appl 70, 525–542 (2014). https://doi.org/10.1007/s11042-013-1376-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1376-1

Keywords

Search

Navigation