Abstract
Online social media services such as Flickr and Zooomr allow users to share their images with the others for social interaction. An important feature of these services is that the users manually annotate their images with the freely-chosen tags, which can be used as indexing keywords for image search and other applications. However, since the tags are generally provided by grassroots Internet users, there is still a gap between these tags and the actual content of the images. This deficiency has significantly limited tag-based applications while, on the other hand, poses a new challenge to the multimedia research community. It calls for a series of research efforts for processing these unqualified tags, especially in making use of content analysis techniques to improve the descriptive power of the tags with respect to the image contents. This paper provides a comprehensive survey of the technical achievements in the research area of content-based tag processing for social images, covering the research aspects on tag ranking, tag refinement and tag-to-region assignment. We review the research advances for each topic and present a brief suggestion for future promising directions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Currently Flickr offers two options in the ranking for tag-based image search. One is “most recent”, which ranks the most recently uploaded images on the top and the other is “most interesting”, which ranks the images by “interestingness”, a measure that takes click-through, comments, etc, into account, as stated in http://www.flickr.com/explore/interesting.
References
Anderson P (2007) What is web 2.0? Ideas, technologies and implications for education. JISC technical report
Cao L, Fei-Fei L (2007) Spatially coherent latent topic model for concurrent segmentation and classification of objects and scenes. In: IEEE ICCV
Chen L, Xu D, Tsang I (2010) Tag-based web photo retrieval improved by batch mode re-tagging. In: IEEE CVPR
Chen Y, Zhu L, Yuille A, Zhang H-J (2009) Unsupervised learning of probabilistic object models (poms) for object classification, segmentation, and recognition using knowledge propagation. In: TPAMI
Chua T, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) Nus-wide: a real-world web image database from National University of Singapore. In: ACM CIVR
Datta R, Joshi D, Li J, Wang J (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):1–60
Feng S, Lang C, Xu D (2010) Beyond tag relevance: integrating visual attention model and multi-instance learning for tag saliency ranking. In: ACM CIVR
Jeon J, Lavrenko V, Manmatha R. Automatic image annotation and retrieval using cross-media relevance models. In: ACM SIGIR
Jing S, Baluja S (2008) VisualRank: applying pageRank to large-scale image search. TPAMI 30(11):1877–1890
Kennedy L, Chang S-F, Kozintsev I (2006) To search or to label?: predicting the performance of search-based automatic image classifiers. In: ACM MIR
Kennedy L, Slaney M, Weinberger K (2009) Reliable tags using image similarity: mining specificity and expertise from large-scale multimedia databases. In: ACM WSMC
Lee S, Neve W, Ro Y (2010) Image tag refinement along the ‘what’ dimension using tag categorization and neighbor voting. In: IEEE ICME
Lew M, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: state of the art and challenges. TOMCCAP 2(1):1–19
Li X, Snoek C, Worring M (2008) Learning tag relevance by neighbor voting for social image retrieval. In: ACM MIR
Li X, Snoek C, Worring M (2009) Learning social tag relevance by neighbor voting. TMM 11(7):1310–1322
Li X, Snoek C, Worring M (2010) Unsupervised multi-feature tag relevance learning for social image retrieval. In: ACM CIVR
Liu D, Hua X-S, Yang L, Wang M, Zhang H-J (2009) Tag ranking. In: ACM WWW
Liu D, Hua X-S, Wang M, Zhang H-J (2010) Image retagging. In: ACM MM
Liu D, Yan S, Rui Y, Zhang H-J (2010) Unified tag analysis with multi-edge graph. In: ACM MM
Liu X, Cheng B, Yan S, Tang J, Chua T, Jin H (2009) Label to region by bi-layer sparsity priors. In: ACM MM
Lu Y, Zhang L, Tian Q, Ma W (2008) What are the high-level concepts with small semantic gaps? In: IEEE CVPR
Rattenbury T, Good N, Naaman M (2007) Towards extracting flickr tag semantics. In: ACM WWW
Shotton J, Winn J, Rother C, Criminisi A (2006) Textonboost: joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: ECCV
Sigurbjörnsson B, Zwol R (2008) Flickr tag recommendation based on collective knowledge. In: ACM WWW, pp 327–336
Wang Z, Feng H, Yan S, Zhang C (2010) Learning to rank tags. In: ACM CIVR
Weinberger K, Slaney M, Zwol R (2008) Resolving tag ambiguity. In: ACM MM, pp 111–120
Yanai K, Barnard K (2005) Image region entropy: a measure of “visualness” of web images associated with one concept? In: ACM MM
Zha J, Yang L, Mei T, Wang M, Wang Z (2009) Visual query suggestion. In: ACM MM
Zhu G, Yan S, Ma Y (2010) Image tag refinement towards low-rank, content-tag prior and error sparsity. In: ACM MM
Acknowledgements
The authors would like to thank Xirong Li, Xiaobai Liu, and Dr. Guangyu Zhu who contribute the figures illustrating their works introduced in this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, D., Hua, XS. & Zhang, HJ. Content-based tag processing for Internet social images. Multimed Tools Appl 51, 723–738 (2011). https://doi.org/10.1007/s11042-010-0647-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-010-0647-3