Yifang Yin
Roger Zimmermann
Skip Abstract Section
Abstract
Videos are increasingly geotagged and used in practical and powerful GIS applications. However, video search and management operations are typically supported by manual textual annotations, which are subjective and laborious. Therefore, research has been conducted to automate or semi-automate this process. Since a diverse vocabulary for video annotations is of paramount importance towards good search results, this article proposes to leverage crowdsourced data from social multimedia applications that host tags of diverse semantics to build a spatio-temporal tag repository, consequently acting as input to our auto-annotation approach. In particular, to build the tag store, we retrieve the necessary data from several social multimedia applications, mine both the spatial and temporal features of the tags, and then refine and index them accordingly. To better integrate the tag repository, we extend our previous approach by leveraging the temporal characteristics of videos as well. Moreover, we set up additional ranking criteria on the basis of tag similarity, popularity and location bias. Experimental results demonstrate that, by making use of such a tag repository, the generated tags have a wide range of semantics, and the resulting rankings are more consistent with human perception.
- Golnaz Abdollahian and Edward J. Delp. 2009. User generated video annotation using geo-tagged image databases. In Proceedings of the IEEE International Conference on Multimedia and Expo. Google Scholar
Digital Library
- Shane Ahern, Mor Naaman, Rahul Nair, and Jeannie Hui. 2007. World Explorer: Visualizing aggregate data from unstructured text in geo-referenced collections. In Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries. Google ScholarDigital Library
- Morgan Ames and Mor Naaman. 2007. Why We Tag: Motivations for annotation in mobile and online media. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Google ScholarDigital Library
- Sakire Arslan Ay, Roger Zimmermann, and Seon Ho Kim. 2008. Viewable scene modeling for geospatial video search. In Proceedings of the ACM Multimedia Conference. Google ScholarDigital Library
- Lamberto Ballan, Marco Bertini, Alberto Del Bimbo, Marco Meoni, and Giuseppe Serra. 2010. Tag suggestion and localization in user-generated videos based on social knowledge. In Proceedings of the 2nd ACM SIGMM Workshop on Social Media. 3--8. Google ScholarDigital Library
- Lamberto Ballan, Marco Bertini, Alberto Del Bimbo, and Giuseppe Serra. 2011. Enriching and localizing semantic tags in internet videos. In Proceedings of the ACM Multimedia Conference. 1541--1544. Google ScholarDigital Library
- C. Brunsdon, A. S. Fotheringham, and M. Charlton. 2002. Geographically weighted summary statistics a framework for localised exploratory data analysis. Computers Environ. Urban Syst. 26, 6, 501--524.Google ScholarCross Ref
- Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. Google ScholarDigital Library
- Nello Cristianini and John Shawe-Taylor. 2000. An Introduction to Support Vector Machines: and Other Kernel-Based Learning Methods. Cambridge University Press. Google ScholarDigital Library
- A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. Royal Statistical Soc. Series B.Google Scholar
- Martin Ester, Hans-peter Kriegel, Jörg Sander, and Xiaowei Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining.Google ScholarDigital Library
- S. L. Feng, R. Manmatha, and V. Lavrenko. 2004. Multiple Bernoulli relevance models for image and video annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarDigital Library
- Yue Gao, Jinhui Tang, Richang Hong, Qionghai Dai, Tat S. Chua, and Ramesh Jain. 2010. W2Go: A travel guidance system by automatic landmark ranking. In Proceedings of the ACM Multimedia Conference. Google ScholarDigital Library
- Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA data mining software: An update. SIGKDD Explor. Newsl. 10--18. Google ScholarDigital Library
- Claudia Hauff. 2013. A study on the accuracy of Flickr's geotag data. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 1037--1040. Google ScholarDigital Library
- John R. Hershey and Peder A. Olsen. 2007. Approximating the Kullback Leibler divergence between Gaussian mixture models. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Vol. 4. 317--320.Google Scholar
- Richang Hong, Jinhui Tang, Hung-Khoon Tan, Chong-Wah Ngo, Shuicheng Yan, and Tat-Seng Chua. 2011. Beyond search: Event-driven summarization for web videos. ACM Trans. Multimedia Comput. Commun. Appl. 7, 4, 35:1--35:18. Google ScholarDigital Library
- Suradej Intagorn and Kristina Lerman. 2011. Learning boundaries of vague places from noisy annotations. In Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 425--428. Google ScholarDigital Library
- Ramesh Jain and Pinaki Sinha. 2010. Content without context is meaningless. In Proceedings of the ACM Multimedia Conference. Google ScholarDigital Library
- J. Jeon, V. Lavrenko, and R. Manmatha. 2003. Automatic image annotation and retrieval using cross-media relevance models. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 119--126. Google ScholarDigital Library
- Martha Larson, Maria Eskevich, Roeland Ordelman, Christoph Kofler, Sebastian Schmiedeke, and Gareth J. F Jones. 2011a. Overview of MediaEval 2011 rich speech retrieval task and genre tagging task. In Proceedings of the Multimedia Benchmark Workshop.Google Scholar
- Martha Larson, Mohammad Soleymani, Pavel Serdyukov, Stevan Rudinac, Christian Wartena, Vanessa Murdock, Gerald Friedland, Roeland Ordelman, and Gareth J. F. Jones. 2011b. Automatic tagging and geotagging in video collections and communities. In Proceedings of the 1st ACM International Conference on Multimedia Retrieval. 51:1--51:8. Google ScholarDigital Library
- Jing Liu, Bin Wang, Mingjing Li, Zhiwei Li, Weiying Ma, Hanqing Lu, and Songde Ma. 2007. Dual cross-media relevance model for image annotation. In Proceedings of the ACM Multimedia Conference. 605--614. Google ScholarDigital Library
- Xin Lu, Changhu Wang, Jiang M. Yang, Yanwei Pang, and Lei Zhang. 2010. Photo2Trip: Generating travel routes from geo-tagged photos for trip planning. In Proceedings of the ACM Multimedia Conference. Google ScholarDigital Library
- Florent Monay and Daniel G. Perez. 2003. On image auto-annotation with latent space models. In Proceedings of the ACM Multimedia Conference. Google ScholarDigital Library
- Emily Moxley, Jim Kleban, and B. S. Manjunath. 2008. SpiritTagger: A geo-aware tag suggestion tool mined from Flickr. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. Google ScholarDigital Library
- Guo J. Qi, Xian S. Hua, Yong Rui, Jinhui Tang, Tao Mei, and Hong J. Zhang. 2007. Correlative multi-label video annotation. In Proceedings of the ACM Multimedia Conference. Google ScholarDigital Library
- Abu Saleh Md Mahfujur Rahman, M Anwar Hossain, and Abdulmotaleb El Saddik. 2010. Spatial-geometric approach to physical mobile interaction based on accelerometer and IR sensory data fusion. ACM Trans. Multimedia Comput. Commun. Appl. 6, 4, 28:1--28:23. Google ScholarDigital Library
- Tye Rattenbury, Nathaniel Good, and Mor Naaman. 2007. Towards automatic extraction of event and place semantics from Flickr tags. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. Google ScholarDigital Library
- Zhijie Shen, Sakire Arslan Ay, Seon Ho Kim, and Roger Zimmermann. 2011. Automatic tag generation and ranking for sensor-rich outdoor videos. In Proceedings of the ACM Multimedia Conference. Google ScholarDigital Library
- Stefan Siersdorfer, Jose San Pedro, and Mark Sanderson. 2009. Automatic video tagging using content redundancy. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. Google ScholarDigital Library
- Börkur Sigurbjörnsson and Roelof Van Zwol. 2008. Flickr tag recommendation based on collective knowledge. In Proceedings of the ACM Conference on the World Wide Web. Google ScholarDigital Library
- Sergej Sizov. 2010. GeoFolk: Latent spatial semantics in Web 2.0 social media. In Proceedings of the 3rd ACM International Conference on Web Search and Data Mining. 281--290. Google ScholarDigital Library
- Cees G. M. Snoek, Koen E. A. van de Sande, Xirong Li, Masoud Mazloom, Yu-Gang Jiang, Dennis C. Koelma, and Arnold W. M. Smeulders. 2011. The MediaMill TRECVID 2011 semantic video search engine.Google Scholar
- Fabian M. Suchanek, Milan Vojnovic, and Dinan Gunawardena. 2008. Social Tags: Meaning and Suggestions. In Proceedings of the 17th ACM Conference on Information and Knowledge Management. Google ScholarDigital Library
- Bart Thomee and Adam Rae. 2013. Uncovering locally characterizing regions within geotagged data. In Proceedings of the ACM Conference on the World Wide Web. 1285--1296. Google ScholarDigital Library
- Xinmei Tian, Dacheng Tao, and Yong Rui. 2012. Sparse transfer learning for interactive video search reranking. ACM Trans. Multimedia Comput. Commun. Appl. 8, 3, 26:1--26:19. Google ScholarDigital Library
- Kentaro Toyama, Ron Logan, and Asta Roseway. 2003. Geographic location tags on digital images. In Proceedings of the ACM Multimedia Conference. Google ScholarDigital Library
- Lei Wu, Linjun Yang, Nenghai Yu, and Xian S. Hua. 2009. Learning to tag. In Proceedings of the ACM Conference on the World Wide Web. Google ScholarDigital Library
- Rong Yan, Apostol Natsev, and Murray Campbell. 2008. A learning-based hybrid tagging and browsing approach for efficient manual image annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Keiji Yanai, Hidetoshi Kawakubo, and Bingyu Qiu. 2009. A visual analysis of the relationship between word concepts and geographical locations. In Proceedings of the ACM International Conference on Image and Video Retrieval. Google ScholarDigital Library
- Zhijun Yin, Liangliang Cao, Jiawei Han, Jiebo Luo, and Thomas S. Huang. 2011a. Diversified trajectory pattern ranking in geo-tagged social media. In Proceedings of the SIAM International Conference on Data Mining. 980--991.Google Scholar
- Zhijun Yin, Liangliang Cao, Jiawei Han, Chengxiang Zhai, and Thomas Huang. 2011b. Geographical topic discovery and comparison. In Proceedings of the ACM Conference on World Wide Web. 247--256. Google ScholarDigital Library
- Haipeng Zhang, Mohammed Korayem, Erkang You, and David J. Crandall. 2012. Beyond co-occurrence: Discovering and visualizing tag relationships from geo-spatial and temporal similarities. In Proceedings of the ACM International Conference on Web Search and Data Mining. Google ScholarDigital Library
Index Terms
- Spatial-Temporal Tag Mining for Automatic Geospatial Video Annotation
Recommendations
Automatic tag generation and ranking for sensor-rich outdoor videos
MM '11: Proceedings of the 19th ACM international conference on MultimediaVideo tag annotations have become a useful and powerful feature to facilitate video search in many social media and web applications. The majority of tags assigned to videos are supplied by users - a task which is time consuming and may result in ...
SRV-TaGS: An Automatic TAGging and Search System for Sensor-Rich Outdoor Videos
MM '11: Proceedings of the 19th ACM international conference on MultimediaTagging facilitates video search in many social media and web applications. While manual tagging is time consuming, subjective and sometimes inaccurate, auto-tagging facilitated by content-based techniques is compute-intensive and challenging to apply ...
Disinformation in Multimedia Annotation: Misleading Metadata Detection on YouTube
iV&L-MM '16: Proceedings of the 2016 ACM workshop on Vision and Language Integration Meets Multimedia FusionPopularity of online videos is increasing at a rapid rate. Not only the users can access these videos online, but they can also upload video content on platforms like YouTube and Myspace. These videos are indexed by user generated multimedia annotation, ...
Comments