Skip to main content
SpringerLink
Log in

Image auto-annotation via concept interdependency network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the explosive growth of multimedia data such as unlabeled images on the Web, image auto-annotation has been receiving increasing research interest. By automatically assigning a set of concepts to unlabeled images, image retrieval can be performed over labeled concepts. Most existing studies focus on the relations between images and concepts, and ignore the interdependencies between labeled concepts. In this paper, we propose a novel image auto-annotation model which utilizes the concept interdependency network to achieve better image auto-annotation. When a concept and its interdependent concepts have a high co-occurrence frequency in the training set, we consider boosting the chance of predicting this concept if there is strong visual evidence for the interdependent concepts in an unlabeled image. Additionally, we combine the global concept interdependency and the local concept interdependency to enhance the auto-annotation performance. Extensive experiments on Corel and IAPR datasets show that the proposed approach almost outperforms all existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bishop CM, et al. (2006) Pattern recognition and machine learning, vol 1. Springer, New York

    MATH  Google Scholar 

  2. Carneiro G, Chan AB, Moreno PJ, Vasconcelos N (2007) Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans Pattern Anal Mach Int 29(3):394–410

    Article  Google Scholar 

  3. Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Int Syst Technol (TIST) 2(3):27

    Google Scholar 

  4. Chen M, Zheng A, Weinberger K (2013) Fast image tagging. In: Proceedings of the 30th international conference on Machine Learning, pp 1274–1282

  5. Chen PI, Lin SJ, Chu YC (2011) Using google latent semantic distance to extract the most relevant information. Expert Syst Appl 38(6):7349–7358

    Article  Google Scholar 

  6. Choi J, Cho M, Park SH, Kim P (2003) Concept-based image retrieval using the new semantic similarity measurement. In: Computational Science and Its ApplicationsłCCSA 2003. Springer, pp 79–88

  7. Cilibrasi RL, Vitanyi PM (2007) The google similarity distance. IEEE Trans Knowl Data Eng 19(3): 370–383

    Article  Google Scholar 

  8. Cui C, Ma J, Lian T, Wang X, Ren Z (2013) Ranking-oriented nearest-neighbor based method for automatic image annotation. In: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 957–960

  9. Das SR, Panigrahi PK, Das K, Mishra D (2012) Improving rbf kernel function of support vector machine using particle swarm optimization. International Journal

  10. Duygulu P, Barnard K, de Freitas J F, Forsyth DA (2002) Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Computer VisionłECCV 2002. Springer, pp 97–112

  11. Fellbaum C (1998) WordNet. Wiley Online Library

  12. Feng S, Manmatha R, Lavrenko V (2004) Multiple bernoulli relevance models for image and video annotation. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, vol 2. IEEE, pp II–1002

  13. Feng Z, Jin R, Jain A (2013) Large-scale image annotation by efficient and robust kernel metric learning. In: 2013 IEEE International Conference on Computer Vision (ICCV). IEEE, pp 1609–1616

  14. Fu H, Zhang Q, Qiu G (2012) Random forest for image annotation. In: Computer Vision–ECCV 2012. Springer, pp 86–99

  15. Grubinger M, Clough P, Müller H, Deselaers T (2006) The iapr tc-12 benchmark: A new evaluation resource for visual information systems. In: International Workshop OntoImage, pp 13–23

  16. Guillaumin M, Küttel D, Ferrari V (2014) Imagenet auto-annotation with segmentation propagation. International Journal of Computer Vision: pp 1–21

  17. Guillaumin M, Mensink T, Verbeek J, Schmid C (2009) Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In: IEEE 12th International Conference on Computer Vision, 2009. IEEE, pp 309–316

  18. Gruber T (1993) A translation approach to portable ontology specifications. Knowl Acquis 5(2):199–220

    Article  Google Scholar 

  19. Gruber T (2009) Ontology. Encyclopedia of database systems, pp 1963–1965

  20. Hu J, Lam KM (2013) An efficient two-stage framework for image annotation. Pattern Recog 46(3):936–947

    Article  Google Scholar 

  21. Huang Z, Qiu Y (2010) A multiple-perspective approach to constructing and aggregating citation semantic link network. Futur Gener Comput Syst 26(3):400–407

    Article  MathSciNet  Google Scholar 

  22. Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval. ACM, pp 119–126

  23. Jin R, Chai JY, Si L (2004) Effective automatic image annotation via a coherent language model and active learning. In: Proceedings of the 12th annual ACM international conference on Multimedia. ACM, pp 892–899

  24. Jin Y, Khan L, Wang L, Awad M (2005) Image annotations by combining multiple evidence & wordnet. In: Proceedings of the 13th annual ACM international conference on Multimedia. ACM, pp 706–715

  25. Lavrenko V, Manmatha R, Jeon J (2003) A model for learning the semantics of pictures. In: Advances in neural information processing systems, p. None

  26. Liu J, Li M, Liu Q, Lu H, Ma S (2009) Image annotation via graph learning. Pattern Recog 42(2):218–228

    Article  MATH  Google Scholar 

  27. Maji S, Berg AC, Malik J (2008) Classification using intersection kernel support vector machines is efficient. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE, pp 1–8

  28. Makadia A, Pavlovic V, Kumar S (2010) Baselines for image annotation. Int J Comput Vis 90(1):88–105

    Article  Google Scholar 

  29. Manevitz LM, Yousef M (2002) One-class svms for document classification. J Mach Learn Res 2:139–154

    MATH  Google Scholar 

  30. Metzler D, Manmatha R (2004) An inference network approach to image retrieval. In: Image and video retrieval. Springer, pp 42–50

  31. Moran S, Lavrenko V (2014) Sparse kernel learning for image annotation. In: Proceedings of international conference on Multimedia Retrieval. ACM, p 113

  32. Nguyen CT, Kaothanthong N, Tokuyama T, Phan XH (2013) A feature-word-topic model for image annotation and retrieval. ACM Trans Web (TWEB) 7(3):12

    Google Scholar 

  33. Srikanth M, Varner J, Bowden M, Moldovan D (2005) Exploiting ontologies for automatic image annotation. In: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 552–558

  34. Verma Y, Jawahar C (2013) Exploring svm for image annotation in presence of confusing labels. In: Proceedings of the 24th British Machine Vision Conference

  35. Wang C, Zhang L, Zhang HJ (2008) Scalable markov model-based image annotation. In: Proceedings of the 2008 international conference on Content-based image and video retrieval. ACM, pp 113–118

  36. Wang M, Xia X, Le J, Zhou X (2014) Effective automatic image annotation via integrated discriminative and generative models. Inf Sci 262:159–171

    Article  MathSciNet  Google Scholar 

  37. Wang Z, Guan G, Qiu Y, Zhuo L, Feng D (2013) Semantic context based refinement for news video annotation. Multimedia Tools Appl 67(3):607–627

    Article  Google Scholar 

  38. Wei XY, Jiang YG, Ngo CW (2011) Concept-driven multi-modality fusion for video search. IEEE Trans Circ Syst Video Technol 21(1):62–73

    Article  Google Scholar 

  39. Xiang Y, Zhou X, Chua TS, Ngo CW (2009) A revisit of generative model for automatic image annotation using markov random fields. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE, pp 1153–1160

  40. Xie L, Pan P, Lu Y, Wang S, Zhu T, Xu H, Chen D (2013) A two-phase generation model for automatic image annotation. In: 2013 IEEE International Symposium on Multimedia (ISM), pp 155–162

  41. Xu H, Pan P, Lu Y, Xu C, Chen D (2014) Improving automatic image annotation with google semantic link. In: 2014 10th international conference on Semantics, Knowledge and Grids (SKG), pp 177–184

  42. Yavlinsky A, Schofield E, Rüger S (2005) Automated image annotation using global features and robust nonparametric density estimation. In: Image and video retrieval. Springer, pp 507–517

  43. Yu Y, Pedrycz W, Miao D (2013) Neighborhood rough sets based multi-label classification for automatic image annotation. Int J Approx Reason 54(9):1373–1387

    Article  MATH  Google Scholar 

  44. Zhang D, Islam MM, Lu G (2012) A review on automatic image annotation techniques. Pattern Recognit 45(1):346–362

    Article  Google Scholar 

  45. Zhang S, Huang J, Li H, Metaxas DN (2012) Automatic image annotation and retrieval using group sparsity. IEEE Trans on Systems, Man, and Cybernetics, Part B: Cybernetics 42(3):838–849

    Article  Google Scholar 

  46. Zhuge H (2004) The knowledge grid, vol 2012. World Scientific

  47. Zhuge H (2010) Interactive semantics. Artif Intell 174(2):190–204

    Article  Google Scholar 

  48. Zhuge H (2011) Semantic linking through spaces for cyber-physical-socio intelligence: A methodology. Artif Intell 175(5):988–1019

    Article  Google Scholar 

  49. Zhuge H, Sun Y (2010) The schema theory for semantic link network. Futur Gener Comput Syst 26(3):408–420

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by HUST Independent Innovation Research Foundation project (No. 2014QN007). Thanks to the help from my colleagues in National University of Singapore.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, People’s Republic of China

    HaiJiao Xu, Peng Pan, YanSheng Lu & Deng Chen

  2. Electrical and Computer Engineering, National University of Singapore, 21 Lower Kent Ridge Road, Singapore, 119077, Singapore

    ChunYan Xu

Authors
  1. HaiJiao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. ChunYan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. YanSheng Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Deng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Pan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, H., Pan, P., Xu, C. et al. Image auto-annotation via concept interdependency network. Multimed Tools Appl 75, 6237–6261 (2016). https://doi.org/10.1007/s11042-015-2568-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2568-7

Keywords

Search

Navigation