Abstract
Automatic image annotation as a typical multi-label learning problem, has gained extensive attention in recent years owing to its application in image semantic understanding and relevant disciplines. Nevertheless, existing annotation methods share the same challenge that labels annotated on the training images are usually incomplete and unclean, while the need for adequate training data is costly and unrealistic. Being aware of this, we propose a dual low-rank regularized multi-label learning model under a graph regularized semi-supervised learning framework, which can effectively capture the label correlations in the learned feature space, and enforce the label matrix be self-recovered in label space as well. To be specific, the proposed approach firstly puts forward a label matrix refinement approach, by introducing a label coefficient matrix to build a linear self-recovery model. Then, graph Laplacian regularization is introduced to make use of a large number of unlabeled images by enforcing the local geometric structure on both labeled and unlabeled images. Lastly, we exploit dual trace norm regularization on both feature mapping matrix and self-recovery coefficient matrix to capture the correlations among different labels in both feature space and label space, and control the model complexity as well. Empirical studies on four real-world image datasets demonstrate the effectiveness and efficiency of the proposed framework.







Similar content being viewed by others
References
Bao B, Liu G, Xu C, Yan S (2012) Inductive robust principal component analysis. IEEE Trans Image Process 21(8):3794–3800
Bao B, Zhu G, Shen J, Yan S (2013) Robust image analysis with sparse representation on quantized visual features. IEEE Trans Image Process 22(3):860–871
Bucak SS, Jin R, Jain A (2011) Multi-label learning with incomplete class assignments. In: IEEE Conference on computer vision and pattern recognition, pp 2801–2808
Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: International conference on knowledge discovery and data mining, pp 333–342
Carneiro G, Chan AB, Moreno PJ, Vasconcelos N (2007) Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans Pattern Anal Mach Intell 29(3):394–410
Chen M, Zheng A, Weinberger KQ (2013) Fast image tagging. In: International conference on machine learning, pp 1274–1282
Chua TS, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) Nus-wide: a real-world web image database from national university of singapore. In: ACM International conference on image and video retrieval, pp 1–9
Fan J, Shen Y, Yang C, Zhou N (2011) Structured max-margin learning for inter-related classifier training and multilabel image annotation. IEEE Trans Image Process 20(3):837–854
Feng S, Feng Z, Jin R (2015) Learning to rank image tags with limited training examples. IEEE Trans Image Process 24(4):1223–1234
Feng S, Lang C (2017) Graph regularized low-rank feature mapping for multi-label learning with application to image annotation. Multidim Syst Sign Process 26(4):1–22
Feng Z, Jin R, Jain A (2013) Large-scale image annotation by efficient and robust kernel metric learning. In: IEEE International conference on computer vision, pp 1609–1616
Goldberg AB, Zhu X, Recht B, Xu J, Nowak R (2010) Transduction with matrix completion: three birds with one stone. In: International conference on neural information processing systems, pp 757–765
Guillaumin M, Mensink T, Verbeek J, Schmid C (2009) Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In: IEEE International conference on computer vision, pp 309–316
Huang S, Zhou Z (2012) Multi-label learning by exploiting label correlations locally. In: Twenty-sixth AAAI conference on artificial intelligence, pp 949–955
Hwang SJ, Grauman K (2012) Learning the relative importance of objects from tagged images for retrieval and cross-modal search. Int J Comput Vis 100(2):134–153
Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: International ACM SIGIR conference on research and development in informaion retrieval, pp 119–126
Ji S, Ye J (2009) An accelerated gradient method for trace norm minimization. In: International conference on machine learning, pp 457–464
Jian L, Li J, Shu K, Liu H (2016) Multi-label informed feature selection. In: International joint conference on artificial intelligence, pp 1627–1633
Jing L, Yang L, Yu J, Ng MK (2015) Semi-supervised low-rank mapping learning for multi-label classification. In: IEEE International conference on computer vision and pattern recognition, pp 1483–1491
Li B, Xiong W, Wu O, Hu W, Maybank S, Yan S (2015) Horror image recognition based on context-aware multi-instance learning. IEEE Trans Image Process 24(12):5193–5205
Li B, Yuan C, Xiong W, Hu W, Peng H, Ding X, Maybank S (2017) Multi-view multi-instance learning based on joint sparse representation and multi-view dictionary learning. IEEE Trans Pattern Anal Mach Intell 39(12):2554–2560
Li J, Wang J (2003) Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans Pattern Anal Mach Intell 25(9):1075–1088
Li X, Zhao X, Zhang Z, Wu F, Zhuang Y, Wang J, Li X (2015) Joint multilabel classification with community-aware label graph learning. IEEE Trans Image Process 25(1):484–493
Lin Z, Ding G, Hu M, Wang J (2014) Multi-label classification via feature-aware implicit label space encoding. In: International conference on machine learning, pp 325–333
Makadia A, Pavlovic V, Kumar S (2010) Baselines for image annotation. Int J Comput Vis 90(1):88–105
Monay F, Gaticaperez D (2004) Plsa-based image auto-annotation: constraining the latent space. In: ACM International conference on multimedia, pp 348–351
Nesterov Y (1983) A method of solving a convex programming problem with convergence rate \(o(\frac {1}{k^{2}})\). In: Soviet mathematics doklady, pp 372–376
Peng H, Li B, Ling H, Hu W, Xiong W, Maybank SJ (2016) Salient object detection via structured matrix decomposition. IEEE Trans Pattern Anal Mach Intell 39(4):818–832
Putthividhy D, Attias HT, Nagarajan SS (2010) Topic regression multi-modal latent dirichlet allocation for image annotation. In: IEEE International conference on computer vision and pattern recognition, pp 3408–3415
Sang J, Fang Q, Xu C (2017) Exploiting social-mobile information for location visualization. ACM Trans Intell Syst Technol 8(3):39
Sang J, Xu C, Liu J (2012) User-aware image tag refinement via ternary semantic analysis. IEEE Trans Multimedia 14(3):883–895
Toh K-C, Yun S (2009) An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pacific J Optim 6(3):615–640
Wang H, Huang H, Ding C (2009) Image annotation using multi-label correlated green’s function. In: IEEE International conference on computer vision, pp 2029–2034
Wang X, Zhang L, Li X, Ma W (2008) Annotating images by mining image search results. IEEE Trans Pattern Anal Mach Intell 30(11):1919–1932
Wu L, Jin R, Jain A (2013) Tag completion for image retrieval. IEEE Trans Pattern Anal Mach Intell 35(3):716–727
Xu L, Wang Z, Shen Z, Wang Y, Chen E (2014) Learning low-rank label correlations for multi-label classification with missing labels. In: IEEE International conference on data mining, pp 1067–1072
Xu M, Jin R, Zhou Z (2013) Speedup matrix completion with side information: application to multi-label learning. In: Advances in neural information processing systems, pp 2301–2309
Yang Y, Wu F, Nie F, Shen H, Zhuang Y, Hauptmann AG (2012) Web and personal image annotation by mining label correlation with relaxed visual graph embedding. IEEE Trans Image Process 21(3):1339–1351
Yuan Z, Sang J, Xu C, Yan L (2014) A unified framework of latent feature learning in social media. IEEE Trans Multimedia 16(6):1624–1635
Zhang M, Zhou Z (2007) Ml-knn: a lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048
Zhang ML (2011) Lift: multi-label learning with label-specific features. In: International joint conference on artificial intelligence, pp 1609–1614
Zhao F, Guo Y (2015) Semi-supervised multi-label learning with incomplete labels. In: International joint conference on artificial intelligence, pp 4062–4068
Zhao F, Guo Y (2016) Improving top-n recommendation with heterogeneous loss. In: International joint conference on artificial intelligence, pp 2378–2384
Zhao F, Xiao M, Guo Y (2016) Predictive collaborative filtering with side information. In: International joint conference on artificial intelligence, pp 2385–2390
Acknowledgements
This work is supported in part by National Natural Science Foundation of China (61472028, 61502026, 61673048), the Fundamental Research Funds for the Central Universities (2017JBZ108), Beijing Natural Science Foundation (4162048) and the Joint Research Fund for The Ministry of Education of China and China Mobile (MCM20160206).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, X., Feng, S. & Lang, C. Semi-supervised dual low-rank feature mapping for multi-label image annotation. Multimed Tools Appl 78, 13149–13168 (2019). https://doi.org/10.1007/s11042-018-5719-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-5719-9