Abstract
To compensate for incomplete or imprecise tags in training samples, this paper proposes a learning algorithm for the convolutional neural network (CNN) for multi-label image annotation by introducing co-occurrence dependency between tags as a graph Laplacian regularization term. To exploit the co-occurrence dependency, we apply Hayashi’s quantification method-type III to the tags in the training samples and use the distances between the acquired representative vectors to define the weights for graph Laplacian regularization. By introducing this regularization term, the possibility of co-occurrence between tags with high co-occurrence frequency can be increased. To confirm the effectiveness of the proposed algorithm, we have done experiments using Corel5k’s dataset for multi-label image annotation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Flicker. https://www.flickr.com
Instagram. https://www.instagram.com
Ames, M., Naaman, M.: Why we tag: motivations for annotation in mobile and online media. In: Proceedings of the Conference on Human Factors in Computing Systems, pp. 971–980. ACM (2007)
Guillaumin, M., Mensink, T., Verbeek, J., Schmid, C.: TagProp: discriminative metric learning in nearest neighbor models for image auto-annotation. In: 2009 International Conference Computer Vision, pp. 309–316. IEEE (2009)
Barnard, K., Duygulu, P., Forsyth, D., Freitas, N.D., Blei, D.M., Jordan, M.I.: Matching words and pictures. J. Mach. Learn. Res. 3(Feb), 1107–1135 (2003)
Grangier, D., Bengio, S.: A discriminative kernel-based approach to rank images from text queries. IEEE Trans. Pattern Anal. Mach. Intell. 30(8), 1371–1384 (2008)
Hertz, T., Bar-Hillel, A., Weinshall, D.: Learning distance functions for image retrieval. In: 2004 Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, p. II. IEEE (2004)
Makadia, A., Pavlovic, V., Kumar, S.: A new baseline for image annotation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 316–329. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88690-7_24
LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Gong, Y., Jia, Y., Leung, T., Toshev, A., Ioffe, S.: Deep convolutional ranking for multilabel image annotation arXiv:1312.4894 (2013)
Wei, Y., Xia, W., Huang, J., Ni, B., Dong, J., Zhao, Y., Yan, S.: CNN: single-label to multi-label arXiv:1406.5726 (2014)
Xue, X., Zhang, W., Zhang, J., Wu, B., Fan, J., Lu, Y.: Correlative multi-label multi-instance image annotation. In: 2011 IEEE International Conference Computer Vision, pp. 651–658. IEEE (2011)
Guo, Y., Gu, S.: Multi-label classification using conditional dependency networks. In: IJCAI Proceedings-International Joint Conference on Artificial Intelligence, vol. 22, p. 1300 (2011)
Wang, J., Yang, Y., Mao, J., Huang, Z., Huang, C., Xu, W.: CNN-RNN: a unified framework for multi-label image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2285–2294 (2016)
Hayashi, C.: Multidimensional quantification. I. Proc. Jpn. Acad. 30(2), 61–65 (1954)
Benzecri, J.P.: Lanalyse des donnees, tome II. Lanalyse des correspondances (1973)
Nishisato, S.: Analysis of Categorical Data: Dual Scaling and Its Applications. JSTOR (1980)
Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: NIPS, vol. 14, pp. 585–591 (2001)
Watanabe, K., Kurita, T.: Locality preserving multi-nominal logistic regression. In: ICPR, pp. 1–4. IEEE (2008)
Acknowledgement
This work was partly supported by JSPS KAKENHI Grant Number 16K00239.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Mojoo, J., Kurosawa, K., Kurita, T. (2017). Deep CNN with Graph Laplacian Regularization for Multi-label Image Annotation. In: Karray, F., Campilho, A., Cheriet, F. (eds) Image Analysis and Recognition. ICIAR 2017. Lecture Notes in Computer Science(), vol 10317. Springer, Cham. https://doi.org/10.1007/978-3-319-59876-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-59876-5_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59875-8
Online ISBN: 978-3-319-59876-5
eBook Packages: Computer ScienceComputer Science (R0)