Skip to main content
SpringerLink
Log in

Automatic tagging by exploring tag information capability and correlation

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Automatic tagging can automatically label images and videos with semantic tags to significantly facilitate multimedia search and organization. However, most of existing tagging algorithms often don’t differentiate between tags used to describe visual content, and neglect the semantic correlation of the assigned tag set. In this paper, we propose a novel automatic tagging algorithm which tags a test image or video with an Informative and Correlative Tag (ICTag) set. The assigned ICTag set can provide a more precise description of the multimedia object by exploring both the information capability of individual tags and the tag-to-set correlation. Measures to effectively estimate the information capability of individual tags and the correlation between a tag and the candidate tag set are designed. To reduce the computational complexity, we also introduce a heuristic method to achieve efficient automatic tagging. We conduct extensive experiments on the NUS-WIDE web image dataset downloaded from Flickr and the MCG-WEBV web video dataset downloaded from YouTube. The results confirm the efficiency and effectiveness of our proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Ames, M., Naaman, M.: Why we tag: motivations for annotation in mobile and online media. In: Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 971–980 (2007)

  2. Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recogn. 37(9), 1757–1771 (2004)

    Article  Google Scholar 

  3. Cao, J., Zhang, Y.D., Song, Y.C., Chen, Z.N., Zhang, X., Li, J.T.: MCG-WEBV: A Benchmark Dataset for Web Video Analysis. Technical Report, ICT-MCG-09-001, Institute of Computing Technology (2009)

  4. Chen, X.Y., Mu, Y.D., Yan, S.C., Chua, T.S.: Efficient large-scale image annotation by probabilistic collaborative multi-label propagation. In: Proceedings of ACM international Conference on Multimedia, pp. 35–44 (2010)

  5. Chua, T., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: Nus-wide: a real-world web image database from National University of Singapore. In: Proceeding of ACM International Conference on Image and Video Retrieval (2009)

  6. Cui, B., Tung, A.K., Zhang, C., Zhao, Z.: Multiple feature fusion for social media applications. In: Proceedings of the ACM International Conference on Management of Data, pp. 435–446 (2010)

  7. Geng, B., Yang, L., Xu, C., Hua, X.: Collaborative learning for image and video annotation. In: Proceedings of ACM MIR 2008 (2008)

  8. Hindle, A., Shao, J., Lin, D., Lu, J., Zhang, R.: Clustering Web video search results based on integration of multiple features. World Wide Web J. 14(1), 53–73 (2011)

    Article  Google Scholar 

  9. Jin, Y., Khan, L., Wang, L., Awad, M.: Image annotations by combining multiple evidence&WordNet. In: Proceedings of 13th ACM International Conference on Multimedia, pp. 706–715 (2005)

  10. Kennedy, L.S., Chang, S.F., Kozintsev, I.V.: To search or to label: predicting the performance of search-based automatic image classifiers. In: Proceedings of the 14th ACM International Workshop on Multimedia Information Retrieval, pp. 249–258 (2006)

  11. Kilian, Q., Malcolm, S., Roelof, Z., Resolving tag ambiguity. In: Proceeding of the ACM International Conference On Multimedia, pp. 111–120 (2008)

  12. Lei, W., Linjun, Y., Nenghai, Y., Xian-Sheng, H.: Learning to tag. In: Proceedings of the ACM International Conference On World Wide Web, pp. 20–24 (2009)

  13. Lei, W., Steven, C.H., Jin, H.R., Jianke, Z., Nenghai, Y.: Distance metric learning from uncertain side information with application to automated photo tagging. In: Proceedings of ACM Multimedia (2009)

  14. Li, H., Wang, Y., Zhang, D., Zhang, M., Chang, E.Y.: Pfp: parallel fp-growth for query recommendation. In: Proceedings of the 2nd ACM Conference on Recommender Systems, pp. 107–114 (2008)

  15. Li, X., Snoek, C.G., Worring, M.: Learning social tag relevance by neighbor voting. IEEE Trans. Multimedia 11, 1310–1322 (2009)

    Article  Google Scholar 

  16. Li, X.R., Snoek, C.G.M., Worring, M.: Learning tag relevance by neighbor voting for social image retrieval. In: Proceeding of 1st ACM International Conference on Multimedia Information Retrieval, pp. 30–31 (2008)

  17. Li, X., Snoek, C.G.M., Worring, M.: Learning social tag relevance by neighbor voting. IEEE Trans. Multimedia 11(7), 1310–1322 (2009)

    Article  Google Scholar 

  18. Li, X.R., Snoek, C.G.M., Worring, M.: Learning tag relevance by neighbor voting for social image retrieval. In: Proceeding of ACM International Conference on Multimedia Information Retrieval (2008)

  19. Li, J., Wang, J.Z.: Real-time computerized annotation of pictures. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 985–1002 (2008)

    Article  Google Scholar 

  20. Liu, D., Hua, X.-S., Wang, M., Zhang, H.-J.: Image retagging. In: Proceedings of ACM Multimedia, pp. 491–500 (2010)

  21. Liu, J., Li, M., Ma, W.-Y., Liu, Q., Lu, H.: An adaptive graph model for automatic image annotation. In: Proceedings of 14th ACM International Conference on Multimedia, pp. 61–70 (2006)

  22. Liu, D., Wang, M., Hua, X.S., Zhang, H.J.: Tag Ranking. In: Proceeding of the ACM International Conference on World Wide Web, pp. 351–340 (2009)

  23. Liu, J., Wang, B., Li, M.J., Li, Z.W., Ma, W.Y., Lu, H.Q., Ma, S.D.: Dual cross-media relevance model for image annotation. In: Proceedings of ACM Multimedia (2007)

  24. Liu, D., Yan, S., Rui, Y., Zhang, H.-J.: Unified tag analysis with multi-edge graph. In: Proceedings of ACM Multimedia, pp. 25–34 (2010)

  25. Lu, Y., Zhang, L., Tian, Q., Ma, W.-Y.: What are the high-level concepts with small semantic gaps? In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

  26. Mei, T., Wang, Y., Hua, X.-S., Gong, S., Li, S.: Coherent image annotation by learning semantic distance. In: Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR) (2008)

  27. Mishne, G.: AutoTag: a collaborative approach to automated tag assignment for weblog posts. In: Proceedings of the ACM international Conference on World Wide Web, pp. 953–954 (2006)

  28. Moxley, E., Mei, T., Manjunath, B.S.: Video annotation through search and graph reinforcement mining. IEEE Trans. Multimedia 12(3), 184–193 (2010)

    Article  Google Scholar 

  29. Qi, G.-J., Hua, X.-S., Rui, Y., Tang, J., Mei, T., Zhang, H.-J.: Correlative multi-label video annotation. In: Proceedings of ACM Multimedia (2007)

  30. Sarkas, N., Das, G., Koudas, N.: Improved search for socially annotated data. In: Proceeding of 35th International Conference on Very Large Data Bases, pp. 778–789 (2009)

  31. Shen, Y., Fan, J.P.: Leveraging loosely-tagged images and inter-object correlations for tag recommendation. In: Proceedings of ACM Multimedia (2010)

  32. Siersdorfer, S., San Pedro, J., Sanderson, M.: Automatic video tagging using content redundancy. In: Proceedings of ACM SIGIR (2009)

  33. Sigurbjrnsson, B., Van Zwol, R.: Flickr tag recommendation based on collective knowledge. In: Proceedings of 17th ACM International Conference of World Wide Web, pp. 327–336 (2008)

  34. Wang, X.-J., Zhang, L., Li, X., Ma, W.-Y.: Annotating images by mining image search results. IEEE Trans. Pattern Anal. Mach. Intell. 30(11), 1919–1932 (2008)

    Article  Google Scholar 

  35. Wu, F., Han, Y.H., Tian, Q., Zhuang, Y.T.: Multi-label Boosting for image annotation by structural grouping sparsity. In: Proceedings of ACM Multimedia (2010)

  36. Xiang, Y., Zhou, X.D., Liu, Z.T., T-Chua, S., Ngo, C-W.: Semantic context modeling with maximal margin conditional random fields for automatic image annotation. In: Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR) (2010)

  37. Yan, R., Hauprmann, A.: Query expansion using probabilistic local feedback with application to multimedia retrieval. In: Proceedings of the 16th ACM Conference on Conference on information and Knowledge Management, pp. 361–370 (2007)

  38. Yang, Y., Huang, Z., Shen, H.T., Zhou, X.F.: Mining multi-tag association for image tagging. World Wide Web J. 14(2), 133–156 (2011)

    Article  Google Scholar 

  39. Yang, Y., Yang, Y., Huang, Z., Shen, H.T., Nie, F.: Tag localization with spatial correlations and joint group sparsity. In: Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR) (2011)

  40. Yates, R.B., Neto, B.R.: Modern Information Retrieval. ACM Press (1999)

  41. Xu, Z., Fu, Y., Mao, J., Su, D.: Towards the semantic web: collaborative tag suggestions. In: Proceedings of Collaborative Web Tagging Workshop at ACM International Conference On World Wide Web, pp. 56–65 (2006)

  42. Zhou, X., Wang, M., Zhang, Q., Zhang, J., Shi, B.: Automatic image annotation by an iterative approach: incorporating keyword correlations and region matching. In: Proceedings of ACM CIVR (2007)

  43. Zhu, Z.-H., Zhang, M.-L.: Multi-instance multi-label learning with application to scene classification. In: Proceedings of NIPS (2006)

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Software Development Environment, Beihang University, Beijing, 100191, China

    Xiaoming Zhang & Zhoujun Li

  2. School of Computer Science and Engineering, Beihang University, Beijing, 100191, China

    Xiaoming Zhang & Zhoujun Li

  3. Beijing Key Laboratory of Network Technology, BeiHang University, Beijing, 100191, China

    Xiaoming Zhang & Zhoujun Li

  4. School of Information Technology & Electrical Engineering, University of Queensland, Building 78, St Lucia Campus, Brisbane, Australia

    Zi Huang, Heng Tao Shen & Yang Yang

Authors
  1. Xiaoming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heng Tao Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhoujun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heng Tao Shen.

Additional information

The preliminary version of this paper has appeared as a short paper in APWEB 2011.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, X., Huang, Z., Shen, H.T. et al. Automatic tagging by exploring tag information capability and correlation. World Wide Web 15, 233–256 (2012). https://doi.org/10.1007/s11280-011-0132-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-011-0132-6

Keywords

Search

Navigation