Skip to main content
SpringerLink
Log in

An efficient refinement algorithm for multi-label image annotation with correlation model

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

With the explosively rising popularity of photography devices, collections of personal digital images are growing rapidly both in number and size. There is an increasing desire to effectively index and search these images to meet user requirements. The content-based image retrieval (CBIR) system facilitates effective image indexing and retrieval according to image features. However, the semantic gap between the low-level visual features and high-level semantic concepts hinders further development. Image annotation is a solution intended to resolve the CBIR system inadequacies. However, there are two problems with the annotation. (1) It is very difficult to represent an image using only a few keywords; (2) the manual annotation process is very subjective, ambiguous, and incomplete. This paper focuses on refining image annotation to cluster the most representative keywords, as the annotations to image with a small semantic gap. We propose the Hierarchical_Twin Rings algorithm to refine the quality of annotations in order to close the well-known semantic gap problem. Moreover, we present another Centroid-based Convergence method of automatically assigning relevant multi-keywords to a user specified image which could greatly improve the retrieval accuracy and fast response requirement. The key contributions of our work areas follows: (1) Weintroduce the problem of the mining of representative image keywords as the annotation for image indexing and retrieval from a large set of image collection. (2) We use Bayesian framework to integrate the image and image annotation into a unifiedframework. (3) Our formulation allows one to refine the relevant annotations of an image and remove redundant annotations. We evaluated the performance of our algorithm by means of images collected from Flickr, the photo sharing website. Our experimental results show that the Hierarchical_Twin Rings algorithm is a realistic and effective method for multi-label image annotation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. http://www.flickr.com

  2. Wang, X. J., Zhang, L., Jing, F., & Ma, W. Y. (2006). AnnoSearch: Image auto-annotation by search. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. doi:10.1109/CVPR.2006.58.

  3. Wang, C.H., Jing, F., Zhang, L., & Zhang, H. J. (2006). Scalable search-based image annotation of personal images. In Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval. doi:10.1145/1178677.1178714.

  4. Wang, C.H., Jing, F., Zhang, L., & Zhang, H.J. (2006). Image annotation refinement using random walk with restarts. In Proceedings of the 14th annual ACM international conference on Multimedia. doi:10.1145/1180639.1180774.

  5. Aboulmagd, H., El-Gayar, N., Onsi, H., et al. (2009). A new approach in content-based image retrieval using fuzzy. Journal of Telecommunication Systems, 40(1–2), 55–66.

    Article  Google Scholar 

  6. Jing, F., Wang, C.H., Yao, Y.H., Deng, K.F., Zhang, L., & Ma, W.Y. (2006). IGroup: Web image search results clustering. In Proceedings of the 14th annual ACM international conference on Multimedia. doi:10.1145/1180639.1180720.

  7. Srinivasa, K. D. (2009). Content based image retrieval. Journal of Computer Sciences and Telecommunications, 6(23), 10–27.

    Google Scholar 

  8. Zhou, T.H., Heo, B.M., Wang, L., Lee, Y.K., Chai, D.J., & Ryu, K.H. (2008). A prototype of multimedia metadata management system for supporting the integration of heterogeneous sources. In Proceedings of the 4th international conference on Intelligent Computing. doi:10.1007/978-3-540-87442-3_135.

  9. Zhang, M. L., Zhou, Z. H., et al. (2007). Ml-knn: A lazy learning approach to multi-label learning. Journal of Pattern Recognition, 40(7), 2038–2048.

    Article  Google Scholar 

  10. Spyromitros, E., Tsoumakas, G., & Vlahavas, I. (2008). An empirical study of lazy multilabel classification algorithms. In Proceedings of the 5th Hellenic Conference on Artificial Intelligence. doi:10.1007/978-3-540-87881-0_40.

  11. Wang, L., Zhou, T.H., Shon, H.S., Lee, Y.K., & Ryu, K.H. (2010). Extract and maintain the most helpful wavelet coefficients for continuous K-nearest neighbor queries in stream processing. In Proceedings of the 6th International Conference on Intelligent Computing. doi:10.1007/978-3-642-14831-6_48.

  12. Frome, A., Sha, F., Singer, Y., & Malik, J. (2007). Learning globally-consistent local distance functions for shape-based image retrieval and classification. In Proceedings of the IEEE International Conference on Computer Vision. doi:10.1109/ICCV.2007.4408839.

  13. Ramakrishnan, S., El Emary, I. M. M., et al. (2009). Comparative study between traditional and modified probabilistic neural networks. Journal of Telecommunication Systems, 40(1–2), 67–74.

    Article  Google Scholar 

  14. Zhou, X.D., Wang, M., Zhang, Q., Zhang, J.Q., & Shi, B.L. (2007). Automatic image annotation by an iterative approach: incorporating keyword correlations and region matching. In Proceedings of the 6th ACM International Conference on Image and Video Retrieval. doi:10.1145/1282280.1282284.

  15. Nasierding, G., & Kouzani, A.Z. (2010). Empirical study of multi-label classification methods for image annotation and retrieval. In Proceedings of the International Conference on Digital Image Computing: Techniques and Applications. doi:10.1109/DICTA.2010.113.

  16. Liu, D., Yan, S. C., Hua, X. S., Zhang, H. J., et al. (2011). Image retagging using collaborative tag propagation. Journal of IEEE Transactions on Multimedia, 13(4), 702–712.

    Article  Google Scholar 

  17. Wang, M., Zhou, X. D., & Chua, T. S. (2008). Automatic image annotation via local multi-label classification. In Proceedings of the International Conference on Content-Based Image and Video Retrieval. doi:10.1145/1386352.1386359.

  18. Nasierding, G., Tsoumakas, G., & Kouzani, A. Z. (2009). Clustering based multi-label classification for image annotation and retrieval. In Proceedings of the IEEE International Conference on Systems Man and Cybernetics. doi:10.1109/ICSMC.2009.5346902.

  19. Sigurbjörnsson, B., & Zwol, R.V. (2008). Flickr tag recommendation based on collective knowledge. In Proceeding of the 17th International Conference on World Wide Web. doi:10.1145/1367497.1367542.

  20. Wu, L., Yang, L.J., Yu, N.H., & Hua, X.C. (2009). Learning to tag. In Proceedings of the 18th International Conference on World Wide Web. doi:10.1145/1526709.1526758.

  21. Singh, M., Curran, E., & Cunningham, P. (2009). Active learning for multi-label image annotation. Technical Report UCD-CSI-2009-01.

  22. Hanbury, A. (2008). A survey of methods for image annotation. Journal of Visual Languages and Computing, 19(5), 617–627.

    Article  Google Scholar 

  23. Munkhdalai, T., Li, M.J., Kim, T.W., Namsrai, O.E., Jeong, S.P., Shin, J.P., & Ryu, K.H. (2012). Bio named entity recognition based on co-training algorithm. In Proceedings of the 26th International Conference on Advanced Information Networking and Applications Workshops. doi:10.1109/WAINA.2012.75.

  24. Carneiro, G., Chan, A. B., Moreno, P. J., Vasconcelos, N., et al. (2007). Supervised learning of semantic classes for image annotation and retrieval. Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3), 394–410.

    Article  Google Scholar 

  25. Makadia, A., Pavlovic, V., & Kumar, S. (2008). A new baseline for image annotation. In Proceedings of the 10th European Conference on Computer Vision. doi:10.1007/978-3-540-88690-7_24.

  26. Choi, K., Toh, K. A., Byun, H., et al. (2011). An efficient incremental face annotation for large scale web services. Journal of Telecommunication Systems, 47(3–4), 197–214.

    Article  Google Scholar 

  27. Kennedy, L.S., Chang, S.F., & Kozintsev, I.V. (2006). To search or to label?: predicting the performance of search-based automatic image classifiers. In Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval. doi:10.1145/1178677.1178712.

  28. Chua, T. S., Tang, J. H., Hong, R. C., Li, H. J., Luo, Z. P., & Zheng, Y. T. (2009). NUS-WIDE: a real-world web image database from National University of Singapore. In Proceeding of the ACM International Conference on Image and Video Retrieval. doi:10.1145/1646396.1646452.

  29. Ames, M., & Naaman, M. (2007). Why we tag: motivations for annotation in mobile and online media. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. doi:10.1145/1240624.1240772.

  30. Liu, D., Hua, X.S., Yang, L.J., Wang, M., & Zhang, H.J. (2009). Tag ranking. In Proceedings of the 18th International Conference on World Wide Web. doi:10.1145/1526709.1526757.

  31. Wang, L., Lee, Y.K., & Ryu, K.H. (2008). Supporting top-k aggregate queries over unequal synopsis on internet traffic streams. In Proceedings of the 10th Asia-Pacific Web Conference on Progress in WWW Research and Development. doi:10.1007/978-3-540-78849-2_59.

  32. Liu, Y. H., Ong, K. L., Lee, V. C. S., Chen, Y. P. P., et al. (2011). Processing multiple image streams for real-time monitoring of parking lots. Journal of Computer, Consumer and Control (IJ3C), 1(1), 23–32.

    Google Scholar 

Download references

Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (No. 2013R1A2A2A01068923), Export Promotion Technology Development Program, Ministry of Agriculture, Food and Rural Affairs (No. 114083-3), and the Science and Technology Plan Projects of Jilin city (No. 201464059).

Author information

Authors and Affiliations

  1. Department of Information Engineering, Northeast Dianli University, Jilin, China

    Ling Wang

  2. Database/Bioinformatics Laboratory, College of Electrical and Computer Engineering, University of Chungbuk, Cheongju, Korea

    Tie Hua Zhou & Keun Ho Ryu

  3. Electronics and Telecommunications Research Institute, Daejeon, Korea

    Yang Koo Lee

  4. Department of Software, College of Electrical and Computer Engineering, University of Chungbuk, Cheongju, Korea

    Kyung Joo Cheoi

Authors
  1. Ling Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tie Hua Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang Koo Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kyung Joo Cheoi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Keun Ho Ryu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Keun Ho Ryu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, L., Zhou, T.H., Lee, Y.K. et al. An efficient refinement algorithm for multi-label image annotation with correlation model. Telecommun Syst 60, 285–301 (2015). https://doi.org/10.1007/s11235-015-0030-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-015-0030-9

Keywords

Search

Navigation