Abstract
Automatic image annotation is a challenging problem due to the label-image-matching, label-imbalance and label-missing problems. Some research tried to address part of these problems but didn’t integrate them. In this paper, an adaptive weighted distance method which incorporates the CNN (convolutional neural network) feature and multiple handcrafted features is proposed to handle the label-image-matching and label-imbalance issues, while the K nearest neighbors framework is improved by using the neighborhood with all labels which can reduce the effects of the label-missing problem. Finally, experiments on three benchmark datasets (Corel-5k, ESP-Game and IAPRTC-12) for image annotation are performed, and the results show that our approach is competitive to the state-of-the-art methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Carneiro, G., Chan, A.B., Moreno, P.J., Vasconcelos, N.: Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 394–410 (2007)
Chen, M., Zheng, A., Weinberger, K.: Fast image tagging. In: Proceedings of the 30th International Conference on Machine Learning, pp. 1274–1282 (2013)
Duygulu, P., Barnard, K., Freitas, J.F.G., Forsyth, D.A.: Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002). doi:10.1007/3-540-47979-1_7
Feng, S., Manmatha, R., Lavrenko, V.: Multiple Bernoulli relevance models for image and video annotation. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, pp. II:1002–II:1009. IEEE (2004)
Gong, Y., Jia, Y., Leung, T., Toshev, A., Ioffe, S.: Deep convolutional ranking for multilabel image annotation. arXiv:1312.4894 (2013)
Gu, Y., Qian, X., Li, Q., Wang, M., Hong, R., Tian, Q.: Image annotation by latent community detection and multikernel learning. IEEE Trans. Image Process. 24(11), 3450–3463 (2015)
Guillaumin, M., Mensink, T., Verbeek, J., Schmid, C.: TagProp: discriminative metric learning in nearest neighbor models for image auto-annotation. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 309–316. IEEE (2009)
He, Y., Wang, J., Kang, C., Xiang, S., Pan, C.: Large scale image annotation via deep representation learning and tag embedding learning. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 523–526. ACM (2015)
Kalayeh, M.M., Idrees, H., Shah, M.: NMF-KNN: Image annotation using weighted multi-view non-negative matrix factorization. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 184–191. IEEE (2014)
Liu, W., Tao, D.: Multiview Hessian regularization for image annotation. IEEE Trans. Image Process. 22(7), 2676–2687 (2013)
Makadia, A., Pavlovic, V., Kumar, S.: A new baseline for image annotation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 316–329. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88690-7_24
Murthy, V.N., Maji, S., Manmatha, R.: Automatic image annotation using deep learning representations. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 603–606. ACM (2015)
Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)
Su, F., Xue, L.: Graph learning on k nearest neighbours for automatic image annotation. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 403–410. ACM (2015)
Tariq, A., Foroosh, H.: Feature-independent context estimation for automatic image annotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1958–1965 (2015)
de Weijer, J., Schmid, C.: Coloring local feature extraction. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 334–348. Springer, Heidelberg (2006). doi:10.1007/11744047_26
Verma, Y., Jawahar, C.V.: Image annotation using metric learning in semantic neighbourhoods. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 836–849. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33712-3_60
Von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 319–326. ACM (2004)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 818–833. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10590-1_53
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Li, J., Yuan, C. (2016). Automatic Image Annotation Using Adaptive Weighted Distance in Improved K Nearest Neighbors Framework. In: Chen, E., Gong, Y., Tie, Y. (eds) Advances in Multimedia Information Processing - PCM 2016. PCM 2016. Lecture Notes in Computer Science(), vol 9916. Springer, Cham. https://doi.org/10.1007/978-3-319-48890-5_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-48890-5_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48889-9
Online ISBN: 978-3-319-48890-5
eBook Packages: Computer ScienceComputer Science (R0)